Towards a Multimodal Sensorimotor Coordination Based Object Recognition System

We present a recognition system which learns to recognize objects based on multimodal sensorimotor coordination. The sensorimotor coordination is generated through interaction with the environment. The system uses a learning architecture which is composed of reactive and deliberative layers. The reactive layer consists of a database of behaviors that are modulated to produce a desired behavior. We have implemented in the learning architecture an object manipulation behavior inspired by the concept that infants learn about their environment through manipulation [1]. While manipulating objects, the agent records both proprioceptive data and exteroceptive data. Both of these types of data are combined and statistically analyzed in order to extract important parameters that distinctively describe the object being manipulated. This data is then clustered using the standard k-means algorithm and the resulting clusters are labeled. The labeling is used to train a radial basis function network for classifying the clusters. The performance of the system has been tested on a kinematically complex walking robot, and it has been found that the trained neural network is able to classify objects even when only partial sensory data is available to the system. Our preliminary results demonstrate that this method can be effectively used in a robotic system which learns from experience about its environment.