Automatic classification of question difficulty level: Teachers´ estimation vs. students´ perception

The accurate estimation of the difficulty level of the questions posed to students is essential to help them to learn more effectively and efficiently. However, it is agreed that teachers usually fail to identify the correct difficulty level of the questions, according to the answers and final scores obtained by their students. Thus, this paper examines the ability of teachers for categorizing questions by difficulty level, comparing it with the students´ perception and the measures obtained by an expert system of question automatic classification. The results show that students perceive questions more difficult than teachers, except for the harder ones. In addition, teachers are only lightly more accurate (closer to the expert system), in spite of the general students´ tendency to overestimate the difficulty level of less difficult questions. Although no general conclusions can be obtained about behavior and accuracy of teachers and students when they analyze the difficulty of learning material, the provided analysis could be very valuable for teachers in order to detect unclear problem statements and students´ misconceptions.