A learning and forgetting algorithm in associative memories: results involving pseudo-inverses

The authors develop a design technique for associative memories with learning and forgetting abilities via artificial feedback neural networks. The method utilizes the theory of large-scale interconnected dynamical systems, instead of the usual energy methods. Networks synthesized by this design method are capable of learning new patterns as well as forgetting old patterns without recomputing the entire interconnection matrix. The method, in which the properties of pseudo-inverse matrices are used to iteratively solve systems of linear equations, provides significant improvements over the outer product method and the projection learning rule. Several specific examples are given to illustrate the strengths and weaknesses of the methodology