How many more images do we need? Performance prediction of bootstrapping for image classification

Motivated by the recently introduced scalable concept detection challenge that requires classifiers for hundreds or even thousands of concepts, the objective of this work is to predict the cases where the enhancement of an initial classifier with additional training images is not expected to provide significant improvements. To facilitate this objective, we need a model for predicting the performance gain of a bootstrapping process prior to actually applying it. In order to train this model, we propose two features; the initial classifier´s maturity (i.e. how close is the current hyperplane to the optimal) and the oracle´s reliability (i.e. how reliable is the oracle in providing the correct labels of new training data). Thus, the contribution of our work is on proposing a method that is able to exploit the correlation between the expected performance boost and these two indicators. As a result, we can considerably improve the scalability properties of such bootstrapping processes by concentrating on the most prominent models and thus reducing the overall processing load.