eAssessment of open questions: An educator’s perspective

The use of language in free text answers provides an excellent means for assessment, but it is hard work to mark this type of material. The automatic assessment of open questions can be applied successfully to carefully written literal text explicit and literal text implicit questions, both categories very frequent in engineering. Existing automatic marking systems exhibit results comparable to manual marking, but most of them fall short of educatorspsila expectations. Teachers like to keep control of marking, judge by themselves unusual answers, correct their own model answer on the fly. Hence, computer assisted assessment (CAA) is preferred to fully automatic assessment. The inherent ambiguity of natural language and the difficulties of knowledge representation add to the constraints imposed by teacherspsila requirements. However, there is still space for a system which is both useful and practicable. This paper reviews the types of questions apt for automatic marking and their usefulness in engineering, examines some representative existing automatic marking systems, states requirements of teachers, reviews state of the art possibilities and puts forward some ideas for the design of a computer assisted open question marking system based on a controlled language for answer writing and knowledge representation comparison for assessment.