Improvements in feature vector selection and parameter optimisation for continuous gesture recognition

Gesture recognition has attracted significant interest due to diverse potential applications, including: hand writing recognition, robot control and human-computer interfaces. This paper identifies and addresses three shortcomings in current approaches to feature vector selection and parameter optimisation for continuous gesture recognition. First, in selecting the final feature vector, researchers typically analyse only a small subset of possible feature combinations; however, the limited subset is likely to omit the optimum feature vector. Second, selection of the final feature vector is based on performance in isolated recognition; however, the final feature vector may not perform adequately in continuous recognition. No protocol currently exists to evaluate and select the final feature vector in continuous recognition mode, thus a novel scoring system is developed. Finally, optimisation of the number of states in the Hidden Markov Models (HMMs) and the number of clusters (k-means clustering) is performed independently, ignoring any possible interdependency. To investigate and address these shortcomings, a gesture recognition system geared towards sign language interpretation is designed. The system is tested on a 9-word gesture vocabulary, and subsequent analysis confirms the above conjectures: first, the optimum feature vector cannot be intuitively predicted and must be determined through rigorous analysis; second, selecting the final feature vector in continuous mode improved the accuracy score by 5.85 % and the perfect sentence recognition by 47.2 %; finally, optimising the number of states and number of clusters simultaneously improved the accuracy score by 3.0 % and the perfect sentence recognition by 11.1%.