Learning with Hidden Information

In many classification problems, there exists additional information which is available during training but not available during testing. In this paper we denote such information as hidden information, and study how to incorporate it to improve the learning performance. Despite its importance, learning with hidden information has not attracted enough attention from the field and existing work in this area remains limited. In this paper we make improvements from two perspectives. First, unlike the related work, we propose a general framework to capture hidden information, which is not limited to a specific type of classifier but is widely applicable to different classifiers. Second, borrowing the tool of Bootstrap widely used in statistics, we are able to numerically quantify the benefits and identify the most useful hidden information. Experiments on both digit and object recognition demonstrate the effectiveness of the proposed approach.