Building representations from fusions of multiple views

A robot sensing system is described that uses multiple sources of information to construct an internal representation of its environment. Initially, object models are used to form the basic representations. These are modified by processes that operate on sequences of sensory information, obtained from sensors that move about in the environment. Two representations are constructed. One is a description of the spatial layout of the environment, represented as an octree, while the other is an object- and feature-based representation. The system handles both expected and unexpected objects, and attempts to register its internal representation with the external world using a variety of predictive, sensory-processing, and matching procedures.