Combination of Multiple Retrieval Systems Using Rank-Score Function and Cognitive Diversity

Combining multiple retrieval systems is a commonly used method to improve the retrieval performance. However, it is still a challenging problem to figure out when and how the combined system can perform better than its individual systems. In this paper, we study these issues by using an information fusion paradigm: Combinatorial Fusion Analysis (CFA). TREC datasets are used as our experiment data. We measure the cognitive diversity between different individual systems by using a rank-score characteristic (RSC) function. Our results demonstrate that: 1) The performance of combination of p systems does not always increase with p, 2) Rank combination is better than score combination in particular when RSC diversity between two individual systems is large enough, and 3) combination of two systems can improve performance only if the two individual systems have relative good performance and are diverse.