Tough assessor or Santa Claus: rationalizing the assessment process

A procedure is described that allows quantitative analysis of assessment results to determine if they are statistically representative of the performance of a student cohort. This procedure provides a method of determining to some degree, whether variations in statistical outcomes of a group are due to the assessment process being unrepresentative or due to natural statistical variation. Further, the procedure provides a means of determining the magnitude of correction or scaling of assessment results to partially correct assessment bias whilst still maintaining a measure of variation that can be ascribed to cohort performance. Whilst the procedure does not cater for all the wide variety of possible sources of variation, including nonhomogeneity of student performance, it does move the judgement of reliability of an assessment outcome, a considerable measure away from purely subjective judgement. The use of an aggregate of individual student performance indices derived from prior work, together with proper assessment of statistical significance dependent on the number of students in the cohort, provides a quantifiable measure upon which the results may be assessed for veracity.