Neuron clustering for mitigating catastrophic forgetting in feedforward neural networks

Catastrophic forgetting is a fundamental problem with artificial neural networks (ANNs) in which learned representations are lost as new representations are acquired. This significantly limits the usefulness of ANNs in dynamic or non-stationary settings, as well as when applied to very large datasets. In this paper, we examine a novel neural network architecture which utilizes online clustering for the selection of a subset of hidden neurons to be activated in the feedforward and back propagation passes. It is shown that such networks are able to effectively mitigate catastrophic forgetting. Simulation results illustrate the advantages of the proposed network with respect to other schemes for addressing the memory loss phenomenon.