Adaptive Patch Features for Object Class Recognition with Learned Hierarchical Models

We present a hierarchical generative model for object recognition that is constructed by weakly-supervised learning. A key component is a novel, adaptive patch feature whose width and height are automatically determined. The optimality criterion is based on minimum-variance analysis, which first computes the variance of the appearance model for various patch deformations, and then selects the patch dimensions that yield the minimum variance over the training data. They are integrated into each level of our hierarchical representation that is learned in an iterative, bottom-up fashion. At each level of the hierarchy, pairs of features are identified that tend to occur at stable positions relative to each other, by clustering the configurational distributions of observed feature co-occurrences using expectation-maximization. For recognition, evidence is propagated using nonparametric belief propagation. Discriminative models are learned on the basis of our feature hierarchy by combining a SVM classifier with feature selection based on the Fisher score. Experiments on two very different, challenging image databases demonstrate the effectiveness of this framework for object class recognition, as well as the contribution of the adaptive patch features towards attaining highly competitive results.