Title :
Correlation between human assessment of essays and ROUGE evaluation of essays´ summaries
Author :
Latif, Seemab ; Wood, Mary McGee ; Nenadic, Goran
Author_Institution :
Sch. of Comput. Sci., Univ. of Manchester, Manchester, UK
Abstract :
In this paper we have addressed the qualitative (human evaluation) and quantitative (ROUGE) evaluation of computer generated summaries of the students´ essays. The experimental results show that there is a positive high correlation between ROUGE scores and human assessment of the essays (human assigned marks). We have also found out that human evaluation of the automatic summaries positively correlates with the human assessment of the essays. These correlations can be used to classify students´ essays into broad bands of quality.
Keywords :
classification; educational administrative data processing; information retrieval; text analysis; ROUGE evaluation; automatic classification; automatic summarization; classify student essay; computer generated summary; experimental result; human assessment; human assigned mark; human evaluation; information retrieval; positive correlation; qualitative evaluation; quantitative evaluation; source document; text length; Computer science; Current measurement; Gold; Humans; Measurement standards; Natural language processing; System testing;
Conference_Titel :
Natural Language Processing, 2009. SNLP '09. Eighth International Symposium on
Conference_Location :
Bangkok
Print_ISBN :
978-1-4244-4138-9
Electronic_ISBN :
978-1-4244-4139-6
DOI :
10.1109/SNLP.2009.5340933
