Relaxation based matching of clusters of keypoints from scale-invariant feature transform on multiple frames of buildings

In recognition of urban buildings, conventional SIFT-based matching is highly prone to error when a large number of similar keypoints are extracted from the repetitive structure of a building. We propose a relaxation based matching of clusters of SIFT keypoints, followed by multi-frame verification. Two-step mean shift clustering is successful in grouping semantically homogeneous keypoints into a cluster. Relaxation based matching significantly enhances the reliability of correspondence, by preserving the structural consistency over matched clusters. The correctness of cluster matching can be verified by multi-frame matching. We tested the proposed methods on several images of buildings of the University of Seoul and achieved significant improvement in precision ratio and recall ratio.