Emotioncy: A Source of Rater Bias

Assessment is a multifaceted process affected by a wide range of psychological states and personality traits of raters. Rater effects, defined as the construct-irrelevant variation associated with rater characteristics, are critical to the reliability and validity of writing assessment (Bachman et al., 1995; Kunnan, 2000). This study set out to investigate the effects of emotioncy on rater’s variability. Emotioncy has been defined as the emotional potency of words (Pishghadam, Tabatabayan Navari, 2013). To objectifying this aim, 50 raters, selected conveniently, assessed translation of an English text translated by 8 advanced EFL learners both holistically and analytically. Raters’ emotioncy degree toward text’s vocabulary was assessed using emotioncy scale (Pishghadam, Baghaee, Seyednozadi, 2016).The relationship between emotioncy and holistic and analytic ratings was investigated using Pearson product-moment correlation coefficient. Preliminary analyses were performed to ensure no violation of the assumptions of normality, linearity and homoscedasticity. Results revealed that there was a strong positive relationship between emotioncy and holistic rating, (r=.54 n= 50, p .001). The relationship between emotioncy and analytic rating was weak, (r=.23 n= 50, p .001). Findings will have implications and applications for language assessment and rater training.