The Factors Influencing Iranian Untrained Raters Rating Oral Proficiency of EFL Learners

This study attempts to identify factors that influence raters’ performance in rating oral proficiency of learners; that is, in assigning scores which factors they attend to and the features of the speech sample which influence or impress them. An ignorance of such factors may lead to a limited and limiting description of Iranian untrained raters’ cognition. To this aim, two groups of participants were selected. The first were 16 language learners of different proficiency level and both genders. The language learners were performed a monologue and their productions were audio- recorded. The second group of participants were 32 English language teachers as untrained raters. They were selected based on their volunteer participation and care was taken to select from both genders and varying teaching experiences. The raters rated the speech samples obtained from the language learners. Each rater, individually, listened to the samples and assigned a score of 1 to 6 as delineated in the CEFR. No instruction, framework or scale were presented to the raters and they had to rate based on their own knowledge and personal judgment. The qualitative analysis detected both overt factors and covert factors. It was also revealed that the raters both attend to relevant factors like fluency, accuracy, complexity and irrelevant factors to English oral proficiency construct like voice quality, personality of the learners or use of communication strategies. Cognitive psychology theories like selective attention, divided attention etc. were used to justify the raters’ locus of attention.