Retrieval of arbitrary 3D objects from robot observations

We have studied the problem of retrieval of arbitrary object instances from a large point cloud data set. The context is autonomous robots operating for long periods of time, weeks up to months and regularly saving point cloud data. The ever growing collection of data is stored in a way that allows ranking candidate examples of any query object, given in the form of a single view point cloud, without the need to access the original data. The top ranked ones can then be compared in a second phase using the point clouds themselves. Our method does not assume that the point clouds are segmented or that the objects to be queried are known ahead of time. This means that we are able to represent the entire environment but it also poses problems for retrieval. To overcome this our approach learns from each actual query to improve search results in terms of the ranking. This learning is automatic and based only on the queries. We demonstrate our system on data collected autonomously by a robot operating over 13 days in our building. Comparisons with other techniques and several variations of our method are shown.