Semantic Object Maps for robotic housework - representation, acquisition and use

In this article we investigate the representation and acquisition of Semantic Objects Maps (SOMs) that can serve as information resources for autonomous service robots performing everyday manipulation tasks in kitchen environments. These maps provide the robot with information about its operation environment that enable it to perform fetch and place tasks more efficiently and reliably. To this end, the semantic object maps can answer queries such as the following ones: “What do parts of the kitchen look like?”, “How can a container be opened and closed?”, “Where do objects of daily use belong?”, “What is inside of cupboards/drawers?”, etc. The semantic object maps presented in this article, which we call SOM⁺, extend the first generation of SOMs presented by Rusu et al. [1] in that the representation of SOM⁺ is designed more thoroughly and that SOM⁺ also include knowledge about the appearance and articulation of furniture objects. Also, the acquisition methods for SOM⁺ substantially advance those developed in [1] in that SOM⁺ are acquired autonomously and with low-cost (Kinect) instead of very accurate (laser-based) 3D sensors. In addition, perception methods are more general and are demonstrated to work in different kitchen environments.