A Graph Modeling Strategy for Multi-touch Gesture Recognition

In most applications of touch based human computer interaction, multi-touch gestures are used for directly manipulating the interface such as scaling, panning, etc. In this paper, we propose using multi-touch gesture as indirect command, such as redo, undo, erase, etc., for the operating system. The proposed recognition system is guided by temporal, spatial and shape information. This is achieved using a graph embedding approach where all previous information are used. We evaluated our multi-touch recognition system on a set of 18 different multi-touch gestures. With this graph embedding method and a SVM classifier, we achieve 94.50% recognition rate. We believe that our research points out a possibility of integrating together raw ink, direct manipulation and indirect command in many gesture-based complex application such as a sketch drawing application.