Essentials of the Probability Model of the Multiple Choice Test

Professors should understand the statistical performance of the examinations they give. The presented probability model for multiple choice tests (MCTs) is known by psychometricians as the simple knowledge or random guessing model, but is not well known outside this community. Yet it gives a good picture of the accuracy with which state of knowledge may be estimated. The knowledge state of a student with respect to the subject at hand may be characterized by the single number p (0lesples1) which is a parameter in a binomial pmf model for the test score. The model accounts for the possibility of random guessing at a solution if the student does not know the answer. We analyze the MCT for both individuals, and populations (classrooms). We consider how the number of test questions and possible solutions affects accuracy in estimating p, and the consequences of using scoring with a penalty for wrong answers. Real classroom data are presented, and justify modeling the distribution of p in a class room via the beta pdf instead of the Gaussian pdf