Scene Matching between a Map and a Hand Drawn Sketch Using Spatial Relations

The goal of this work is to determine the object correspondence between a sketched map and the scene depicted by the sketch, e.g., as represented by an occupancy grid map (OGM) built by a robot. We describe a novel method based on spatial relations for accomplishing this task. Our method is based on using the histogram of forces as scene descriptors. We generate a correspondence map between two scene descriptors and evaluate its confidence. From this map, we generate a one-to-one object correspondence map for the two scenes such that the object correspondence confidence value is maximized. Challenges include the fact that the two scenes may differ in terms of the shape and size of the objects, their orientation, and the objects might be shifted due to translation. The approach is evaluated using several hand drawn sketches that were collected as a part of a user study. We believe that the ability to perform scene matching will make our sketch interface more robust and easier to use, thereby providing us with a more intuitive way of communicating with the robots