Using noise to form a minimal overcomplete basis

The authors previously (1998) developed an extension of a principal component analysis artificial neural network which we have linked to the statistical technique of factor analysis. We have shown that the resulting network can identify the independent components of visual scenes. We now show that, in cases where the factor analysis network identifies factors of greater number than the inherent dimensionality of the input space, the addition of noise leads to an optimally sparse representation of the input data which we link to a minimal overcomplete basis. We show that in cases in which the data set is not itself inherently sparse, the method induces a very sparse description of the data set