Averaged and decorrelated neural networks as a time-series predictor

We study the effect of removing temporal structure in the prediction error. We observe that networks which are not optimally trained, exhibit strong temporal structure in their prediction error, which can be eliminated using linear regression. This elimination improves performance significantly, but does not lead to the best performance which is achieved by training networks until they do not exhibit any such temporal structure. The improvement in performance of ensemble net averaging does not affect possible temporal structure of the error, thus averaging can be performed before or after temporal structure removal. We demonstrate these findings on the sunspot data set