Associating spoken commands with multiple human users in a dynamic environment

This paper presents a method for combining human tracking information with recognised sound commands extracted from localised and separated sound sources, enabling spoken commands to be placed in the context of an inhabited, dynamic environment. Humans are tracked using a laser range finder based system which extracts and tracks leg profiles from the raw range data. A 32 channel microphone array localises (angular) and separates different sound sources in the environment and feeds segmented sounds into a speech recognition system. Human tracking position estimates, sound source localisation results, and recognised commands are associated by probabilistic matching over time of the angle of observation from both the command sound source and the human position estimates. Experiments are conducted verifying the command recognition system and the ability to associate commands with humans in the case of a single human giving commands from multiple locations, and also multiple humans giving commands from multiple locations.