Assessing the assessment — Insights into CS1 exams

The Open University of Israel (OUI) is an institution of higher education which has an open admissions policy and is based primarily on distance learning. The CSI course at the OUI is similar to introductory computing courses at other universities in Israel and worldwide. Over the years, we have composed some tens of CSI final exams and collected data related to the exams. Since there is a high dropout rate for this introductory course, we decided, among other things, to analyze the data examining the different types of questions and the different topics the questions deal with, trying to find out whether the situation can be improved. We examined 3,646 individual exams, collected from 25 final exams administered between 2008 and 2012. The objective was to attempt to identify the difficulties the students faced and to provide us with guidelines for writing final exams in CSI which reflect better the material covered, becoming a fairer assessment instrument and decreasing the failure rate.