Sparse coding of symbolically represented motion patterns and top-down activation based on an associative memory model

As has been claimed by Barlow[2], and reported by some recent neuro-physiological researches, at higher levels in the hierarchy of representations in the brain, sparse coding is adopted. Sparse coding is a kind of neural representation in which a very small number of neurons fire selectively. Because of the small overlaps, the codes have the property of uniform metric, which is very different from the physically sensed continuous patterns. It is also known to be efficient in memory capacity, energy consumption and combinatorial computation. Then the problem is, how sparse codes representing concepts can be generated from accumulated episodic memories that are inevitably complex distributed sensory-motor patterns. We propose here a system that generates sparse codes of concepts of motions from accumulated feature vectors of observed motion patterns, by extending our previous research[9]. We apply to this problem an associative memory dynamics model with a self-organizing nonmonotonic activation function, which automatically finds out the hierarchical cluster structures in the stored data. Based on our analysis of the dynamics of this model, we design an output function for the attractors, which can generate the sparse codes of the symbols of motion patterns.