On projections of Gaussian distributions using maximum likelihood criteria

Generative statistical models with a very large number of parameters are frequently used in real-world data applications, such as large-vocabulary speech recognition (LVCSR). Complex models are needed in order to capture the ubiquitous variability in the observed signal, but data sparsity causes significant problems in their training. One way of dealing with data sparsity is to perform dimensionality reduction of the observed features, with the goal of reducing the model parameter space without sacrificing performance. When the data are Gaussian distributed, the dimensionality reduction can be done efficiently using the maximum likelihood criterion; this leads to the heteroscedastic linear discriminant analysis (HLDA), which is a natural extension of linear discriminant analysis (LDA) to the case where the class-conditional Gaussians have unequal covariance matrices. A further extension of HLDA to multiple transforms (MLDA) can also be tackled efficiently. This paper presents the theory behind HLDA and MLDA, and demonstrates their performance with synthetic data.