Multilevel vision based spatial reasoning for robotic tasks

A model-based image understanding and spatial reasoning system (SRS) has been integrated with a robot planner to provide a testbed for autonomously sensing and responding to complex but well-structured environments. The entire system operates in such a way that the robot planner requests state information for entities on a task panel from the SRS. State information may consist of the location or orientation of a latch or door on the panel or it may reflect a particular function, such as whether a latch is open or closed. Examples which illustrate the operation of the entire system with actual robotic and vision hardware are provided. The advantages and disadvantages of active/passive markers and teleoperation within the context of the integrated system are also discussed