Boosted Hybrid DNN/HMM System Based on Correlation-Generated Targets

In current DNN/HMM hybrid systems, the DNN models are trained by the 1-of-V targets which are obtained by the Viterbi-based forced-alignment. The states are viewed as unrelated and isolated. In fact, some phonemes are acoustically similar. Especially for Chinese, as a tonal language, its number of similar pairs is quadrupled. To add the similarity information between states into the model training, the correlation-generated targets are investigated in DNN modeling. For each frame, besides the target state from the forced-alignment, other states which are similar to this state will be assigned to nonzero values. The similarity degrees between the states are measured through calculating the correlation. In the paper, different methods of generating correlation matrix were investigated, and details of the implementation with the correlation matrix were described. On the task for Mandarin conversational speech recognition in customer-service domain, the experiments showed that the hybrid DNN/HMM System based on correlation-generated targets achieved consistent improvements with different amounts of training data.