Vision-based scene representation for 3D interaction of service robots

In this paper, we present a new approach for scene understanding by an autonomous robot, using imaging sensors only. Our goal is to build in real-time a 3D representation of the scene which is suitable for further 3D interaction. We assume the scene is composed of a mix of known and unknown objects. We present a strategy to automatically build online a consistent 3D representation from a set computer vision algorithms results. We especially focus on 2 important aspects: the constraints and choices of each parts of the system, as well as the global layout of these parts, and the final step where all information are merged together to build a unique 3D representation. Results on real scene and system performance are finally showed to validate the proposed approach