Algorithms for optimal linear combinations of neural networks

Recently, several techniques have been developed for combining neural networks. Combining a number of trained neural networks may yield better model accuracy, without requiring extensive efforts in training the individual networks or optimizing their architecture. However, since the corresponding outputs of the combined networks approximate the same physical quantity (or quantities), the linear dependency (collinearity) among these outputs may affect the estimation of the optimal combination weights for combining the networks, resulting in a combined model which is inferior to the apparent best network. In this paper, we present two algorithms for selecting the component networks for the combination in order to reduce the ill effects of collinearity, thus improving the generalization ability of the combined model. Experimental results are included