Uncertainty-based active ranking for document retrieval

One of the main problems in information retrieval is ranking documents according to their relevance to userspsila queries. Learning to rank is considered as a promising approach for addressing the issue. However, like many other supervised approaches, one of the main problems with learning to rank is the lack of labeled data, as well as labeling instances to create a rank model is time-consuming and costly. Thus, it is beneficial to minimize the number of labeled instances. In this paper, we bring the idea of active learning into ranking problem, and propose a new active ranking approach for document retrieval, referred to as Active RSVM. Specifically, we present an uncertainty- based query function to estimate the uncertainty of each instance, decide which instances can provide more information for the ranker and reduce the labeling cost. Experimental results on two real-world datasets show that our proposed active ranking algorithm can reduce the labeling cost greatly without decreasing the ranking accuracy.