DocumentCode :
3099812
Title :
Robust neural network online learning in time-variant regression models
Author :
Briegel, Thomas ; Tresp, Volker
Author_Institution :
Dept. of Inf. & Commun., Siemens AG, Munich, Germany
fYear :
1999
fDate :
36373
Firstpage :
186
Lastpage :
194
Abstract :
We consider robust online learning in time-variant neural network regression models. Using a state space representation for the neural network´s weight evolution in time we derive weight estimates by maximizing posterior modes via the Fisher scoring algorithm. By taking the family of densities as the output error cost function we get a robust error measure suitable for handling additive outliers. Fisher scoring was implemented using a forward backward pass of fixed length through the data set for every time step resulting in so-called online smoothing algorithms. Furthermore, we derive an EM-type algorithm for approximate maximum likelihood estimation of unknown hyperparameters. Our experiments show that online posterior mode weight smoothing outperforms standard online methods like online backpropagation and extended Kalman filtering, both for Gaussian measurements and non-Gaussian measurements with additive outliers
Keywords :
learning (artificial intelligence); maximum likelihood estimation; neural nets; smoothing methods; state-space methods; Fisher scoring algorithm; Gaussian measurements; additive outliers; approximate maximum likelihood estimation; forward backward pass; nonGaussian measurements; online smoothing algorithms; output error cost function; robust error measure; robust neural network online learning; state space representation; time-variant regression models; weight estimates; weight evolution; Backpropagation algorithms; Cost function; Density measurement; Maximum likelihood estimation; Measurement standards; Neural networks; Robustness; Smoothing methods; State estimation; State-space methods;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Neural Networks for Signal Processing IX, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop.
Conference_Location :
Madison, WI
Print_ISBN :
0-7803-5673-X
Type :
conf
DOI :
10.1109/NNSP.1999.788137
Filename :
788137
Link To Document :
بازگشت