On the transformation mechanisms of multilayer perceptrons with sigmoid activation functions for classifications

This paper studies the transformation mechanisms of multilayer perceptrons with sigmoid activation functions for classifications. The viewpoint is presented that in the input spaces the hyperplanes determined by the hidden basis functions with values of 0 do not play the role of separate hyperplanes, and furthermore such "hyperplanes" do not certainly go through the marginal regions between different classes. The number of hidden units is only related to the number of categories and the sample distribution shapes. The rank of output matrix of hidden units should be taken as the basis for pruning or growing the hidden nodes. As a result, an empirical formula for optimally determining the number of hidden neurons is proposed. Finally, two examples are given to verify it.