A self-learning sensor fusion system for object classification

We propose a learning system for object classification which fuses information from a camera, a radar and a localization unit. The system is illustrated in application to categorization of objects on a highway. The system learns not only prior to its deployment in a supervised mode but also on-board a vehicle during its operation in a self-learning mode. The radar guides a selection of candidate images provided by the camera for subsequent analysis by our learning method. The Multilayer Inplace Learning Network (MILN) is used to distinguish between representations of different objects. Radar information gets coupled with navigational information for accurate localization of objects during self-learning. One of the MILN layers helps to resolve labeling conflicts when localization is not sufficient. A Multi-Resolution MILN which uses higher-resolution levels to reinforce training of lower-resolution levels is also proposed for improved performance when dealing with a wide range of distances to objects.