Variation aware control for reliability

It has been proved that there is a bias-variance-covariance trade-off among the trained neural network ensembles. In this paper, extra learning on random data points was proposed to control the variations of the correlations in the negative correlation learning (NCL). Without the control of the correlations, NCL might have arbitrary values on the unknown data points after learning too much on the training data points. With learning on random data points, the individual neural networks in an ensemble trained by NCL could become even more different by having the lower overlapping rates. Such lower overlapping rates imply that learning on random data could control the variation of the correlations among the individual neural networks. It is necessary to have such variation awareness in learning when the correlations have a great impact on the performance of the learned ensemble.