Video-based on-line handwriting recognition

The use of handwriting provides a natural way of interacting with small portable computers. However, in order to capture handwritten text. online, special input devices are necessary. Therefore, M.E. Munich & P. Perona (1996) proposed to use visual input for pen-based computers. Writing can then be performed on ordinary paper, and pen trajectories are automatically extracted from image sequences recorded during the writing process. On the basis of this work, we developed a complete video-based online handwriting recognition system. We will present the techniques applied for pen tracking, pre-processing, feature extraction, and statistical modeling and recognition. Evaluation results on a writer-independent unconstrained handwriting recognition task demonstrate that the inherent limitations of the video-based approach can be compensated using robust modeling combined with adaptation techniques