A Novel Method to Recognize Complex Dynamic Gesture by Combining HMM and FNN Models

Recognition of dynamic gesture is an important task for gesture-based human-computer interaction. A novel HMM-FNN model is proposed in this paper for the modeling and recognition of complex dynamic gesture. It combines temporal modeling capability of hidden Markov model, and ability of fuzzy neural network for fuzzy rule modeling and fuzzy inference. Complex dynamic gesture has two important properties: its motion can be decomposed and usually being defined in a fuzzy way. By HMM-FNN model, dynamic gesture is firstly decomposed into three independent parts: posture changing, 2D motion trajectory and movement in Z-axis direction, and each of part is modeled by a group of HMM models which represent all fuzzy classes it possibly belongs to. The likelihood probability of HMM model to observation sequence is considered as fuzzy membership for FNN model. In our method, high dimensional gesture feature is transformed into several low dimensional features, which leads to the reduction of model complexity. By means of fuzzy inference, it achieves a higher recognition rate than conventional HMM model. Besides, human´s experience can be taken advantaged to build and optimize model structure. Experiments show that the proposed approach is an effective method for the modeling and recognition of complex dynamic gesture