Speaker turn tracking with mobile microphones: Combining location and pitch information

This paper considers the problem of using binaural microphones to track speakers in a situation where the microphones are themselves in motion (i.e. due to listener head movement). We present a general framework that applies particle filtering to combine sequential interaural time difference (ITD) cues with noisy sensor motion data. The framework is demonstrated in a meeting scenario applied to a moving-listener version of a speaker-diarization task. The paper extends previous work by investigating two potentially complementary ways of exploiting pitch track estimates in this framework, either, i) informing the time points at which speaker turn changes may occur, or ii) improving the ITD estimates by allowing integration over spectro-temporal regions grouped by pitch. Experiments using real meeting scenario recordings, made with in-ear binaural microphones, show that the latter approach leads to large and significant reductions in diarization error rate.