Human-defined, machine-optimized - gesture recognition using a mixed approach

Wearable computing devices are now mainstream. Many such devices have capable MEMS sensors that can be exploited for recognizing dynamic, in-the-air gestures. The somewhat limited compute capacity and battery life of today´s devices requires a computationally efficient approach to gesture recognition; one that can be effectively used inside an app running on standard, off-the-shelf hardware, such as an Android Smartwatch. The goal of this project is to test the feasibility of this idea. In a two-phased approach, a class of finite state machines (FSM¹) for gesture recognition were first constructed and then the FSM were further tuned for higher accuracy with the help of some training data and a suitable optimization method, in the second phase. A novel approach is presented that leverages techniques from functional programming languages to define rich yet compact FSM. In order to demonstrate effectiveness, a prototype gesture recognition system for an automotive scenario using an Android Smartwatch app was developed. Then the system was tuned using a blended approach that combined a swarm-based evolutionary algorithm, Cultural Algorithms, and human parameter estimates with experimentally derived training data. The Blended approach using Cultural Algorithms achieved a 77% gesture recognition accuracy which is on par with more computationally intensive techniques such as Hidden Markov Models (HMM). The ´functional´ FSM are human defined but machine optimized with Cultural Algorithms. By the blending of the two approaches, an improved balance between computational requirements and recognition accuracy was achieved.