Analysis and pruning of nonlinear auto-association networks

In the paper, an analysis of a three-layer nonlinear auto-association network with linear output neurons and sigmoidal hidden neurons is carried out. Simulations have shown that the hidden layer neurons of this network operate mainly in their linear region. By studying the statistical relations governing the operation of such a network, the nearly linear behaviour of the sigmoidal hidden neurons was verified. Dealing with the network as being totally linear, a pruning algorithm is proposed to find out the minimum number of hidden neurons needed to reconstruct the input data within a certain error threshold. The performance of the pruning algorithm is illustrated with two examples.