A Search for Accountability in Low-stakes Tests: An Evidence-based Approach to Validation of a Local Writing Test

Validation has always been a significant part of assessment in educational settings. However, due to its complexities, most officials prefer to use ready-made tests to avoid validating their own tests. This study validates a screening writing test in an EFL context using an evidence-based validation approach. This study reflects the researcher’s attempts to collect evidence to justify the newly developed test use and interpretation. The participants of this study were 53 Iranian B1 English language learners. To collect the required evidence, the researcher employed a questionnaire to investigate the learners’ cognitive processes that they went through to write the texts. Furthermore, the scores were correlated with the participants’ scores on a First Certificate in English (FCE) writing task to attain the criterion-related validity evidence. Evidence related to scoring validity and consequential validity was also collected by employing simple statistical measures, such as correlation, t-test, and percentage, that are easy for teachers to run. The findings of the study showed the (cognitive, context, scoring, consequential, and criterion-related) validity of the developed test. The pieces of evidence collected in this study supported the validity of the test for the intended use. The present report can enlighten novice teachers as to how to collect a body of evidence to ensure different stakeholders about the validity of their tests. While validation might be perceived as a complicated process, which is not within teachers’ ability, this study can provide a blueprint to guide teachers on how to show their accountability through easy-to-implement steps.