Automatic prediction of item difficulty for short-answer questions

In the construction of evaluation questions, the evaluator faces the problem of choosing what questions are the best to make a distinction between students according to their level of skill or knowledge. The selection of appropriate questions presupposes to know the level of difficulty of each one. It has seen the need to introduce open questions in the tests because these questions evaluate cognitive abilities different from those that are evaluated by closed questions. The selection of these questions for a test involves problems such as high economic costs and risks of confidentiality. This study involved extracting 46 factors of 196 SciEntsBank items. It is done from the texts of the question, the reference response, and cognitive demand. Our aim was to find a method for the automatic difficulty prediction of short-answer questions with some degree of reliability. We got 2.070 features from 46 combined factors for each item. We use KBest supervised method to select the best features. Next, we predicted the difficulty of items using the best features and two types of regression models (linear and support vector machine). Finally, the best model to predict the item difficulty used a support vector machine and the feature of counting keywords.