A novel stream-weight method for the multi-stream speech recognition system

A multi-stream speech recognition system is based on the combination of multiple complementary feature streams. Utilizing the fusion scheme of multi-stream, better performance was achieved in speech recognition system. The stream-weight method plays a very important role in the fusion collaborative scheme. The stream weights should be selected to be proportional to the feature stream reliability and informativeness. The posterior probability estimate is a measure of reliability, and the classification error is a measure of informativeness. The larger separation between class distributions in a given stream implies better discriminative power. The intra-class distances are an estimate of the class variance. The inter- and intra-class distances are combined to yield and estimate of the misclassification error for each stream. An unsupervised stream weight estimation method for multi-stream speech recognition system based on the computation of intra-and inter-class distances in each stream is proposed here. Experiments are conducted using Chinese Academy of Science speech database. Applying the new stream-weigh algorithm, we achieve better fusion performance compared with some traditional fusion methods, and the word error rate was decreased by 6%.