Reducing linear redundancy in neural networks

The authors show how to reduce linear redundancy both in the input vectors and in the weight vectors of one layer of neurons by using two simple one-layer feedforward networks with linear neurons. The main contributions are to show how to use linear dependences in the weight matrix to reduce the number of connections; and to give neuronal tools to optimize another network. A new class of neurons is proposed to reduce the disparity among neurons introduced by this technique. Two simple neural networks are proposed to perform network optimization, showing that linear redundancy can be eliminated from the network within the neural network paradigm