Visual-verbal machine interpreter for sign language recognition under versatile video backgrounds

This research paper is an attempt to create a video background independent sign language recognition (SLR) system. SLR acts as a Machine Interpreter (MI) between a mute person and normal person. One of the key difficulties in sign language recognition is video background of a sign video in which signer is located. Signer is extracted from cluttered back video backgrounds using boundary and prior shape information. Active contours energy function is formulated by amalgamating energy functions from boundary and shape prior elements. Energy minimization for movement of active contour is achieved using Euler- Lagrange equations. Feature vector is constructed from the segmented signer frames using a frame average based pooling function along with the shape inform obtained from active contour. Artificial Neural Network is constructed to classify and recognize gestures from video frames of signers. Compared to traditional methods of sign language recognition, the proposed Visual-Verbal Machine Interpreter (V2MI) for sign language recognition offers a recognition rate of around 93%.