Learning to Rank with Only Positive Examples

Search By Multiple Examples (SBME) is a new search paradigm that allows users to specify their information needs as a set of relevant documents rather than as a set of keywords. In this study, we propose a Transductive Positive Unlabeled learning (TPU learning) based framework for SBME. The framework consists of two steps: 1) identifying potential relevant documents for searching space reduction, and 2) adopting TPU learning methods to re-rank the documents in the new searching space. Using MAP and p@k, we evaluate two state-of-the-art PU learning algorithms and the Rocchio classifier (Rc) for document ranking in the proposed framework. We then adopt the idea of ensemble learning to combine Rc with the two state-of-the-art PU learning algorithms respectively. Experiments conducted on a benchmark dataset show that the ensemble learning based methods lead to a significant improvement in effectiveness.