Completing the cycle: meaningful course evaluations

Good educational practice, to say nothing of standards imposed by every regional accrediting association, mandate that an educational institution perform ongoing evaluations of academic programs. Perhaps the most prevalent method in which schools address this mandate is to use a general-purpose ´smile sheet´ that attempts to measure the student´s satisfaction with the course. This type of evaluation is certainly flawed. Although there have been a number of studies supporting the reliability and validity of these general-purpose instruments, they do not provide the granularity of feedback necessary for the instructor to make decisions regarding changes necessary or desirable in future iterations of the course. This presentation will discuss the results of a study that tested the efficacy of a system for evaluating courses at the design level based upon the test matrix approach commonly used in software engineering. Three questions are addressed: 1) what data are necessary for meaningful design-level assessment; 2) how can that data be gathered and 3) how can that data be interpreted.