Towards a SignWriting recognition system

SignWriting is a writing system for sign languages. It is based on visual symbols to represent the hand shapes, movements and facial expressions, among other elements. It has been adopted by more than 40 countries, but to ensure the social integration of the deaf community, writing systems based on sign languages should be properly incorporated into the Information Technology. This article reports our first efforts toward the implementation of an automatic reading system for SignWiring. This would allow converting the SignWriting script into text so that one can store, retrieve, and index information in an efficient way. In order to make this work possible, we have been collecting a database of hand configurations, which at the present moment sums up to 7,994 images divided into 103 classes of symbols. To classify such symbols, we have performed a comprehensive set of experiments using different features, classifiers, and combination strategies. The best result, 94.4% of recognition rate, was achieved by a Convolutional Neural Network.