Can Motion Segmentation Improve Patch-Based Object Recognition?

Patch-based methods, which constitute the state of the art in object recognition, are often applied to video data, where motion information provides a valuable clue for separating objects of interest from the background. We show that such motion-based segmentation improves the robustness of patch-based recognition with respect to clutter. Our approach, which employs segmentation information to rule out incorrect correspondences between training and test views, is demonstrated empirically to distinctly outperform baselines operating on unsegmented images. Relative improvements reach 50% for the recognition of specific objects, and 33% for object category retrieval.