Visual speech recognition of Modern Classic Arabic language

Viseme-based Visual Speech Recognition (VSR) systems, using Hidden Markov Models (HMM) for phoneme recognition, generally use 3-state left-right HMM for each viseme to recognize. In this article, we propose a novel approach introducing a consonant-vowel detector and using two classifiers: an HMM based classifier for the recognition of the “consonant part” of the phoneme and a classifier for the “vowel part”. The benefits of such an approach include (1) reducing the number of hidden states and (2) reducing the number of HMMs. We tested our method on a limited set of words of the Modern Classic Arabic language and achieved a recognition rate of 81.7%. Moreover, the proposed model is speaker-independent and uses visemes as the basic units, thereby, making it applicable to any set of words of varying size or content.