A PCA Based Visual DCT Feature Extraction Method for Lip-Reading

This paper proposes a PCA based method to reduce the dimensionality of DCT coefficients for visual only lip-reading systems. A three-stage pixel based visual front end is adopted. First, DCT or block-based DCT features are extracted. Second, Principal Component Analysis is applied for dimension reduction. Finally, all the feature vectors are normalized into a uniform scale. This work investigates this three-stage method, comparing with PCA and two DCT based approaches whose features are selected manually. In the latter manner, PCA coefficients are selected according to energy while the reduction of DCT coefficients leans to the left components in the left-top corner. Experiments prove that the dimension reduction task based on PCA does improve the recognition accuracy when the final dimension is below a certain value. They also show that DCT and block-based DCT work similarly for lip reading task, outperforming PCA slightly.