A Coarse-to-Fine Subpixel Registration Method to Recover Local Perspective Deformation in the Application of Image Super-Resolution

In this paper, a coarse-to-fine framework is proposed to register accurately the local regions of interest (ROIs) of images with independent perspective motions by estimating their deformation parameters. A coarse registration approach based on control points (CPs) is presented to obtain the initial perspective parameters. This approach exploits two constraints to solve the problem with a very limited number of CPs. One is named the point-point-line topology constraint, and the other is named the color and intensity distribution of segment constraint. Both of the constraints describe the consistency between the reference and sensed images. To obtain a finer registration, we have converted the perspective deformation into affine deformations in local image patches so that affine refinements can be used readily. Then, the local affine parameters that have been refined are utilized to recover precise perspective parameters of a ROI. Moreover, the location and dimension selections of local image patches are discussed by mathematical demonstrations to avoid the aperture effect. Experiments on simulated data and real-world sequences demonstrate the accuracy and the robustness of the proposed method. The experimental results of image super-resolution are also provided, which show a possible practical application of our method.