Visual Gesture Character String Recognition by Classification-Based Segmentation with Stroke Deletion

The recognition of character strings in visual gestures has many potential applications, yet the segmentation of characters is a great challenge since the pen lift information is not available. In this paper, we propose a visual gesture character string recognition method using the classification-based segmentation strategy. In addition to the character classifier and character geometry models used for evaluating candidate segmentation-recognition paths, we introduce deletion geometry models for deleting stroke segments that are likely to be ligatures. To perform experiments, we built a Kinect-based fingertip trajectory capturing system to collect gesture string data. Experiments of digit string recognition show that the deletion geometry models improve the string recognition accuracy significantly. The string-level correct rate is over 80%.