Improvements on bottleneck feature for large vocabulary continuous speech recognition

In this paper, we have proposed three methods to improve the performance of the bottleneck(BN) feature based GMM-HMM system. Firstly, we recommend a new bottleneck feature architecture, namely LBN, which places the bottleneck layer at the last hidden layer instead of the middle, in order to take advantage of the more discriminative and invariant higher layer features. Secondly, we employ the rectified linear units (ReLUs) based DNN as bottleneck feature extractor. Finally, we investigate the sequence discriminative training of bottleneck neural network to achieve more powerful bottleneck feature. We have evaluated our methods in 309-hour Switchboard (SWB) task. Compared with the traditional hybrid DNN-HMM system, our proposed ReLUs based LBN-GMM-HMM system can achieve about 9.7% recognition error rate reduction relatively.