Title :
Robust neural network online learning in time-variant regression models
Author :
Briegel, Thomas ; Tresp, Volker
Author_Institution :
Dept. of Inf. & Commun., Siemens AG, Munich, Germany
Abstract :
We consider robust online learning in time-variant neural network regression models. Using a state space representation for the neural network´s weight evolution in time we derive weight estimates by maximizing posterior modes via the Fisher scoring algorithm. By taking the family of densities as the output error cost function we get a robust error measure suitable for handling additive outliers. Fisher scoring was implemented using a forward backward pass of fixed length through the data set for every time step resulting in so-called online smoothing algorithms. Furthermore, we derive an EM-type algorithm for approximate maximum likelihood estimation of unknown hyperparameters. Our experiments show that online posterior mode weight smoothing outperforms standard online methods like online backpropagation and extended Kalman filtering, both for Gaussian measurements and non-Gaussian measurements with additive outliers
Keywords :
learning (artificial intelligence); maximum likelihood estimation; neural nets; smoothing methods; state-space methods; Fisher scoring algorithm; Gaussian measurements; additive outliers; approximate maximum likelihood estimation; forward backward pass; nonGaussian measurements; online smoothing algorithms; output error cost function; robust error measure; robust neural network online learning; state space representation; time-variant regression models; weight estimates; weight evolution; Backpropagation algorithms; Cost function; Density measurement; Maximum likelihood estimation; Measurement standards; Neural networks; Robustness; Smoothing methods; State estimation; State-space methods;
Conference_Titel :
Neural Networks for Signal Processing IX, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop.
Conference_Location :
Madison, WI
Print_ISBN :
0-7803-5673-X
DOI :
10.1109/NNSP.1999.788137