Sound Source Localization Based on Robot Hearing and Vision

A novel sound source localization system was proposed in this paper. The system is composed of a robot which has a microphone array corresponding to human´s ears and 2-CCD cameras corresponding to human´s eyes. The system is computationally efficient and physically compact, thus suitable for mobile robots. The robot can establish the relationship among the hearing, vision and the sound direction, sound source localization is realized based on time delay of arrival (TDOA) estimation using microphone array, then it can be improved by vision. The method is proved accurate and efficient enough for real-time operation.