Neural networks in noisy environment: a simple temporal higher order learning for feed-forward networks

The convergence of neural networks when the mapping is accompanied by noise is discussed. An average method is proposed for cases in which the network configuration leads to a noisy energy function during the learning. The proposed method features time-windowed weight averaging, which proves efficient in the presence of Gaussian noise. Temporal averaging, rather than increasing the network size, may be chosen in order to avoid adding local minima. The analysis and examples are based on feedforward network architectures. The filtering observed through the networks indicates that neural networks may be used for multidimensional nonlinear filtering