Boosting a multi-linear classifier with application to visual lip reading

Visual lip-reading systems can enhance the speech recognition systems accuracy. Performance of lip-reading systems is not high in comparison with audio speech recognition systems due to overlap of patterns of classes and outliers. Thus, lip reading is complex classification problem which can be solved efficiently using ensemble methods. Multi-linear-Discriminant Analysis (MLDA) is a recently proposed method which has good classification performance on the face recognition problem. In this study, a new method of boosting algorithm based on MLDA and nearest neighbor is proposed for lip reading problems. Additionally, to enhance the classification accuracy a new feature extraction and combination techniques are proposed which can extract useful feature from lip reading image databases. Extracted features of samples are encoded as tensor objects to feed in MLDA learner of the boosting method. Empirical evaluation of the novel boosting method and feature extraction techniques on the M2VTS image database reveals excellent result with respect to other linear and multi-linear algorithms.