Generalization enhancement of feedforward neural networks based on the convergence of shape errors

The generalization of the trained neural networks mainly depends on the shape errors between network desired output hypersurface and actual output hypersurface. Due to its serious limitation in representing the shape errors, the widely used mean squared error model could not guarantee the effective convergence of the shape errors, therefore causes the poor generalization of the trained neural networks. To tackle this problem, the information fusion technique based on network output errors and partial derivative errors, and enhanced convergence technique based on the local equalization of samples output errors are presented in this paper. These new techniques effectively improve neural network generalization through direct utilization of network partial derivative errors and enhancement of local equalization of samples output errors. Actual computation cases demonstrate that these techniques proposed could substantially improve network generalization and learning efficiency, compared with the currently used network training methods.