Managing Uncertainty in Text-to-Sketch Tracking Problems

Text-to-Sketch (T2S) is a class of problems in which geolocation is performed using natural language descriptions of a location or locations as input. This is a challenging problem due to the many sources of uncertainty inherent to the task: there is often syntactic and semantic ambiguity present in the input observations, as well as referential ambiguity when the language used to describe the scene may refer to many possible objects or locations in the world. Tracking problems, in which the Text-to-Sketch paradigm is extended to incorporate multiple locations and movements over a temporal dimension, introduce additional uncertainty. We describe a tool for managing the uncertainty in Text-to-Sketch problems called MUTTS. The MUTTS system combines traditional natural language processing (NLP) tools with algorithms used to manage uncertainty in mobile robot navigation to allow the temporal and geographical constraints in the text to incrementally reduce the overall uncertainty of a subject´s location and produce high quality sketches of the subject´s location and movements over time.