Harris corners in the real world: A principled selection criterion for interest points based on ecological statistics

In this paper, we consider whether statistical regularities in natural images might be exploited to provide an improved selection criterion for interest points. One approach that has been particularly influential in this domain, is the Harris corner detector. The impetus for the selection criterion for Harris corners, proposed in early work and which remains in use to this day, is based on an intuitive mathematical definition constrained by the need for computational parsimony. In this paper, we revisit this selection criterion free of the computational constraints that existed 20 years ago, and also importantly, taking advantage of the regularities observed in natural image statistics. Based on the motivating factors of stability and richness of structure, a selection threshold for Harris corners is proposed based on a definition of optimality with respect to the structure observed in natural images. As a whole, the paper affords considerable insight into why existing approaches for selecting interest points work, and also their shortcomings. We also demonstrate how a proposal that is inspired by the properties of natural image statistics might be applied to overcome these shortcomings.