Scene Text Recognition with a Hough Forest Implicit Shape Model

Scene text recognition is the problem of recognizing text in natural scene images taken in unconstrained manner. Many approaches have been proposed, but most of them utilize character models only in character recognition phase, the last stage of the process. If characteristic of the target character shape is utilized earlier for the text detection and extraction, text localization would be much easier and therefore, text recognition would be more robust. In this paper, we propose a novel scene text recognition approach which fully utilizes models of target characters from the beginning to the end of the recognition process. Each of target character set is modeled with part-based object model called Implicit Shape Model (ISM) to achieve robustness for partial degradation of text objects. Towards this end, we trained a Hough forest which localizes and aggregates character parts to detect characters candidates from an image. The detected character candidates are verified by organizing the most plausible text lines using dynamic programming. As concrete character models are utilized throughout the process, even extremely deformed texts are detected and recognized, which are hardly detected with previous approaches.