Unsupervised Stream Weight Computation in a Segmentaion Task: Application to Audio-Visual Speech Recognition

We propose an efficient algorithm for unsupervised stream weight estimation in a segmentation task. Our method uses only the information carried by the test signal and the trained models. The work is based on results presented previously for the classification problem where it is indicated that the optimal stream weights are inversely proportional to the single stream misclassification error. We approximate this error relation by the intra- and inter-class distance ratio over the measured class distributions. This approach is then generalized to the segmentation problem by computing the distances among all the concerned classes. The proposed unsupervised estimation algorithm is evaluated on a an audio-visual speech recognition task. The obtained performances are comparable to the supervised minimum error training approach, up to a certain SNR level.