A Modified Boosting Based Neural Network Ensemble Method for Regression and Forecasting

Neural Network ensemble is a kind of method could significantly improve the generalization ability of the learning systems compare to the single network, it works by training a finite number of neural networks and then combining their results. Due to its generalization ability, network ensemble is widely used in regression and forecasting. The ways that realize the ensemble is numerous, one of them is boosting. Usually the regression methods based on boosting pay a lot of attention on the decrease for residual, but little on generalization ability. Although boosting itself has the property of resisting overfitting, too much focus on the training error decrease would impact the generalization ability method should own. To achieve further improvement of generalization ability and also the speed of convergence for the algorithm, this paper comes up with a modified neural network ensemble method based on boosting. The advantage of the proposed algorithm is mainly represented on the ability of decreasing the generalization error more effectively with the residual small enough. The paper below describes the improving method in detail and roughly gives the proof of its convergence. Based on the simulation experiments, it was found that this method actually could get comparative small generalization error in a more rapid way.