Homeomorphic manifold analysis: Learning motion features of image sequence for lipreading

Lipreading techniques have shown bright prospects for speech recognition under noisy environments and for hearing-impaired listeners. In this paper, we discuss a feature extraction method based on the homeomorphic manifold analysis for lipreading. Given a set of image sequences, we think there is an underlying low dimensional unified manifold embedded in the visual space, and each image sequence can be considered as a homeomorphic manifold twisted or even self-intersected in this space. In order to extract the motion features, each sequence is embedded and aligned to the unified manifold, and a mapping matrix is learned from the aligned embedding. Then we adopt the two-dimensional linear discriminant analysis on these mapping matrices to achieve low dimensional features. The proposed method is tested on the OuluVS database, and simulation experiments have shown it can achieve quite satisfying results.