Differential Item Functioning of Iranian National University Entrance Examination for Foreign Language: The Effect of Background Knowledge

Validity and fairness as crucial elements of any standard test may be violated when items function differently in the performances of test takers with various characteristics. The present study aimed to use Multidimensional Rasch model to examine the presence of DIF between two groups of test takers in 2011 version of Iranian National University Entrance Examination for Foreign Languages (INUEEFL), a 70-item high stakes language proficiency test. The two groups of participants, i. e., Mathematics (N=1500) and Humanities (N=1500), with equal number of male and female, were randomly selected out of 103121participants. The results indicated five items out of the 70 items showed noticeable or large DIF. The overall main effect size for academic background showed a 0.48 logit differences (moderate DIF) in favor of the Mathematics group. Although it could not be considered as a large difference and the total test was not biased or unfair against any group, even such a small difference on such a high-stakes test may not be acceptable. This study suggests testing authorities to perform DIF and detect misfitting items for having a better interpretation of test results. Since the fairness of a test is a multifaceted issue (Kunnan, 2005), this study suggests that existence of a number of items with large/ noticeable DIF does not necessarily lead to unfair test. This study also suggests checking local independency of items, particularly in multiple choice tests that can potentially decrease guessing chance of test takers.