Document reconstruction and recognition from an image sequence

A new method for reconstructing a document from an image sequence and recognizing contents of the document is described. In this method frame images in the image sequence are acquired as partial sub-images by a moving camera. The reconstruction includes a rather expensive registration process. To make its use practical, the proposed method registers sub-images hierarchically by sub-tree matching of the layout structure. Images of a corresponding character pair are piled using sub-pixel precision displacement and interpolated to restore image quality and resolution. The proposed method increased the character recognition rate by 6.8 points to 93.1% when resolution was 133 dpi. The computational cost was reduced to 1/23 that of registration by conventional matching