Input-side training in constructive neural networks based on error scaling and pruning

This paper presents two new modifications to the input-side training in constructive one-hidden-layer feedforward neural networks (FNNs). One is based on scaling of the network output error to which output of a hidden unit is expected to maximally correlate. Results from extensive simulations of many regression problems are then summarized to demonstrate that constructive FNNs generalization capabilities may be significantly improved by the new technique. The second contribution is a proposal for a new criterion for input-side weight pruning. This pruning technique removes redundant input-side weights simultaneously with the network constructive scheme, leading to a smaller network with comparable generalization capabilities. Simulation results are provided to illustrate the effectiveness of the proposed pruning technique