3D N-best search for simultaneous recognition of distant-talking speech of multiple talkers

A microphone array is a promising solution for realizing hands-free speech recognition in real environments. Accurate talker localization is very important for speech recognition using the microphone array. However, localization of a moving talker is difficult in noisy reverberant environments. Talker localization errors degrade the performance of speech recognition. To solve the problem, we proposed a new speech recognition algorithm which considers multiple talker direction hypotheses simultaneously. The proposed algorithm performs Viterbi search in 3-dimensional trellis space composed of talker directions, input frames, and HMM states. In this paper we describe a new simultaneous recognition algorithm for distant-talking speech of multiple talkers using the extended 3D N-best search algorithm. The algorithm exploits path distance-based clustering and a likelihood normalization technique appeared to be necessary in order to build an efficient system for our purpose. We evaluated the proposed method using reverberated data, which are those simulated by the image method and recorded in a real room. The image method was used to find the accuracy-reverberation time relationship, and real data was used to evaluate the real performance of our algorithm. The Top 3 result of simultaneous word accuracy was 73.02% under 162 ms reverberation time using the image method.