Layered sensor modalities for improved human-robot interaction

In robotic systems developed for urban search and rescue, information overload can cause operator disorientation and missed detection of victims. In most systems, multiple sensor modalities are present on the robot, and the operator is presented with multiple displays of information requiring attention refocus. We propose a method where multiple sensor inputs are layered into a single, integrated visual display. Such a display might eliminate missed detections and alleviate operator disorientation by allowing the operator to naturally focus on a single display with visual cues added to aid in the use of sound and other non-visual sensor modalities. We have conducted initial trials with multiple sensors in the National Institute of Standards and Technology (NIST) reference test arena for mobile robots to validate the concept and aid in the construction of appropriate display modes and operations.