Hybrid HMM/BN LVCSR system integrating multiple acoustic features

In current HMM based speech recognition systems, it is difficult to supplement acoustic spectrum features with additional information such as pitch, gender, articulator positions, etc. On the other hand, dynamic Bayesian networks (DBN) allow for easy combination of different features and make use of conditional dependencies between them. However, lack of efficient algorithms has prevented their application in large vocabulary continuous speech recognition. The hybrid HMM/BN acoustic model, where HMM are used for modeling of temporal speech characteristics and state probability model is represented by BN, provides a trade off solution to the problem. In this paper we describe the HMM/BN acoustic model and LVCSR system built upon this model. In the HMM/BN model, in addition to speech observation variable, state BN has two more discrete variables representing speaker gender and pitch frequency. Evaluation results on WSJ database showed lower word error rate with respect to the same complexity conventional HMM acoustic model when there is enough training data to estimate reliable HMM/BN parameters.