On variable sizes and sigmoid activation functions of multilayer perceptrons

This paper studies the influences of variable scales and sigmoid activation functions on the performances of multi-layer perceptrons. Generally speaking, it is not certainly suitable to normalize the input data or make the sizes of input variables in the range of [0.0, 1.0]. The viewpoint is explained in details according to the theory of support vector machine (SVM). The convergence and generalization abilities of multilayer perceptrons can be evidently improved by means of the following three methods: (A). Enlarge the sizes of the variable components in the range of [0.0, 3.0]. (B). Change the standard sigmoid activation function f(x)=(1+exp(-x))^-1 into f(x)=3(1+exp(-x/3))^-1. (C). Introduce the sum-of-squares weight term W^TW into the error functions. The classification experiment shows that more than a learning round should be done and the perceptron with the best good generalization performance be held back.