Reduction of difference among trained neural networks by re-learning

It is often that the learned neural networks end with different decision boundaries under the variations of training data, learning algorithms, architectures, and initial random weights. Such variations are helpful in designing neural network ensembles, but are harmful for making unstable performances, i.e., large variances among different learnings. This paper discusses how to reduce such variances for learned neural networks by letting them re-learn on those data points on which they disagrees with each other. Experimental results have been conducted on four real world applications to explain how and when such re-learning works.