Lip-reading based on fuzzy language model

Applying language model in lip-reading system can greatly improve the recognition rate. But the traditional statistical language model depended on corpus excessively, so it could not be used in some special occasions with a small vocabulary corpus. In this paper, according to fuzzy mathematical theory, the fuzzy evaluation sets were firstly established. Then the frequencies of words or sentences in the corpus were represented as fuzzy membership vectors. Based on HMM, a novel recognition model HFM (HMM and Fuzzy Language Model) was proposed. Then a small lip-reading system was constructed. Based on corpus online constructed by computational linguistics research lab, institute of applied linguistic, ministry of education, a small vocabulary corpus was selected and established. Experimental results demonstrated that, compared to the lip-reading system using n-gram language model, applying HFM (did not need smoothing), syllable accuracy can be increased by 6.5%, and sentence accuracy by 22.7%. In addition, exploited language model for text flow analysis, rather than blindly text matching, in single video channel the accuracy can be up to 68.7%.