An Extensive Empirical Study of Automated Evaluation of Multi-Document Summarization

This paper discusses an approach to automated evaluation of multi-document summarization by computing the similarities of automated summaries and human summaries and scoring the automated summaries by their similarities to the human ones. Several schemes are used in our experiment, as well as the effects of stop words and stemming. Our method experimental result is compared to Rouge which is based on n-gram. The test materials for experiments are from DUC 2005 corpus. The results show that our novel scheme produces acceptable results, and may avoid some defects of n-gram.