A high fidelity multi-sensor scene understanding system for autonomous navigation

In order for an autonomous military robot to “appropriately” navigate through a complex environment, it must have an in-depth understanding of the immediate surroundings. We have developed a scene understanding system based on a multi-sensor system that uses an “operator-trained” rule-base to analyze the pixel level attributes across the set of diverse phenomenology imaging sensors. Each pixel is registered to range information so we not only know what but where features are in the environment. This three dimensional labeled world model can then be used to control the speed and steering of the vehicle in an appropriate manner. In this paper we discuss our multi-sensor system, the operator trained analysis algorithm called ONAV (opportunistic navigation), and the reactive control algorithm used to control the speed and steering of the vehicle