Determination of the number of redundant hidden units in a three-layered feedforward neural network

Determination of the number of redundant hidden units in a three-layered feedforward neural network trained on a learning data set is described. For this purpose, a linear equation, OW=t, which describes the three-layered feedforward neural network mapping for the training data set is introduced. It is shown that, if rank of the matrix, O, is not full-rank, we can remove "the number of hidden units minus the rank of O plus one" hidden units from the network without any increase of the error of the network for the training data. It is also shown that by using singular value decomposition this approach can be applicable to a full-rank matrix O with little increase of error. Computer experiments show the effectiveness of the approach.