A Neural Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG

The recognition of hand gestures is a challenging task for the high degrees of freedom of hand motion. We develop a virtual reality based driving training system of self-propelled gun (SPG). For this system, a dataglove with 18 sensors is employed to perform some driving tasks such as pressing switches, manipulating steering wheel, changing gears, etc. To accomplish these tasks, some hand gestures must be defined from the dataglove sensors data. A feedforward neural network can represent an arbitrary functional mapping so it is possible to map raw data directly to the required hand gestures. This paper uses BP neural network to recognize the hand patterns which exist in the raw sensor data of the dataglove. A pattern set of 300 hand gestures is used to train and test the neural network. The recognition system achieves good performance. It can be effectively used in our virtual reality training system of SPG to perform various manipulating tasks in a more fast, precise, and natural way