Novel spatio-temporal features for fingertip writing recognition in egocentric viewpoint

In this paper, we propose a novel feature extraction scheme for fingertip writing recognition in the air for egocentric viewpoint. The inherent challenges in the egocentric vision e.g. rapid camera motion and object´s appearance and disappearance in scene may cause the fingertip to be detected in non-uniformly time separated frames. Most existing approaches do not consider this missing temporal information for feature extraction, which could be utilized to improve performance in ego-vision tasks. The novel feature extraction scheme extracts spatio-temporal features from trajectory of hand movement which are used with Hidden Markov Models for classification. The proposed feature set outperforms current trajectory based feature schemes and achieves 96.7% recognition rate on a novel fingertip trajectory dataset.