Object segmentation and learning through feature grouping and manipulation

In this paper we present a method for learning new objects situated in uncontrolled and unstructured environments. Visual information only is usually not sufficient for a reliable segmentation and learning of unknown objects without any a priori information. We propose an approach in which the robot introduces additional information by manipulating the entities in the scene, thus generating sufficient information to identify objects and accumulate knowledge about them. Our approach involves the extraction of local feature ensembles that provide hints about the existence of an object, the generation of pushing movements to confirm or reject the initial hypothesis, and the fusion of features that satisfy the assumed motion constraints. To ensure the robustness of the system, probabilistic methods such as RANSAC (RANdom SAmple Consensus) are used in several computational stages. Our experimental results show that the system is successful at segmenting objects in complex scenes. The segmented features can be accumulated across different views to extract more comprehensive knowledge about the objects.