Soft multiple winners for sparse feature extraction

A simple and computationally inexpensive neural network method for generating sparse representations is presented. The network has a single layer of linear neurons and, on top of it, a mechanism which assigns a winning strength for each neuron. Both input and output are real valued in contrast to many earlier methods, where either input or output must have been binary valued. Also, the sum of winning strengths does not have to be normalized as in some other approaches. The ability of the algorithm to find meaningful features is demonstrated in a simulation with images of handwritten numerals