A multimodal stroke-based predictive input for efficient Chinese text entry on mobile devices

Handwriting input method is particularly useful for languages with a logographic writing system. This paper introduces a multimodal stroked-based predictive input for the Chinese language. The proposed method requires users to write only the first few strokes of each character and the system will intelligently infer the intended characters by making use of contextual information. Specifically, a statistical n-gram language model is used. Motivated by the work on Haptic Voice Recognition, this paper also incorporates voice input as an additional modality to further enhance the prediction accuracy. Empirical simulation results show that the predictive handwriting input method with 3 initial strokes outperforms the predictive Pinyin input method. Further improvements can be obtained by considering both the initial stroke order and the corresponding stroke layout. Finally, with voice overlay, the proposed multimodal stroke-based predictive input method achieved more than 85% and 95% 1-best prediction accuracies with 2 and 3 initial strokes respectively.