Data sampling based ensemble acoustic modelling

In this paper, we propose a novel technique of using cross validation (CV) data sampling to construct an ensemble of acoustic models for conversational speech recognition. We further propose using hierarchical Gaussian mixture model (HGMM) and repartition training data to increase the ensemble size and diversity. The proposed methods are found to work well together for ensemble acoustic modeling. We also evaluated the quality of the ensemble acoustic models by using the measures of classification margin, average correct score and variance of correct score. We have found that the ensemble of acoustic models increases the margin and the average correct score, and reduces the variance. We compared the performance of our proposed method with a recently reported method of CV expectation maximization (CVEM) for single acoustic models. Our experimental results on a telemedicine automatic captioning task showed that the proposed ensemble acoustic modeling has led to significant improvements in word recognition accuracy.