An efficient hidden node reduction technique for multilayer perceptrons

An efficient method which yields an appropriate number of hidden nodes by iteratively removing unnecessary hidden nodes is proposed. The criterion for selecting the removable nodes is derived from the analysis of information transformation of individual neurons and the network as a whole. A well-trained multilayer perception (MLP) that has a sufficient number of hidden nodes is used. A hidden node is selected with an output weight of the smallest magnitude, and it is removed from the hidden layer. After the removal, additional training is performed. This procedure is iterated until the network performance is not affected severely. The method has been applied to some problems, including handwritten digit recognition, resulting in a significant reduction of hidden nodes