Multi-network system for sensory integration

The aim of our work is to provide a better understanding of multisensory interactions. Starting from an hypothesis of cognitive psychology, we propose a model of multimodal associative memory that integrates all the modality-specific information. The modular architecture consists of different neutral networks that cooperate for modeling both modality-specific low-level recognition and multi-modal high-level identification. A version with three perceptive modalities has been implemented and tested. Experiments confirm the validity of hypothesis on the functional architecture, since the model can simulate a good identification even if one or two modalities are not available. Other realistic phenomena can be observed on the model, such as evocation of mental images or a behavior similar to the McGurk effect