Title :
A study on the simple penalty term to the error function from the viewpoint of fault tolerant training
Author :
Haruhiko, Takase ; Hidehiko, Kita ; Terumine, Hayashi
Author_Institution :
Dept. of Electr. & Electron. Eng., Mie Univ., Japan
Abstract :
We discussed training algorithm for multi-layer neural networks to enhance fault tolerance of the trained networks. In our previous paper, we proposed adding a simple penalty term to the error function for BP algorithm. The penalty term is a simple polynomial (sum of n-th power of weights). It is also introduced for another purpose (structural training). In this paper, we discuss about the effect of the term, especially the effect of its exponent. Through some experiments and discussions, we conclude that the change of the parameter n brings drastic change of its effect. For small n, the training works as the structural training. For large n, the training enhances the fault tolerance of trained networks.
Keywords :
backpropagation; error analysis; fault tolerance; neural nets; polynomials; BP algorithm; error function; fault tolerant training; multilayer neural networks; penalty term; polynomial term; structural training network; Acceleration; Artificial neural networks; Continuous wavelet transforms; Electronic mail; Fault tolerance; Multi-layer neural network; Neural networks; Neurofeedback; Output feedback; Polynomials;
Conference_Titel :
Neural Networks, 2004. Proceedings. 2004 IEEE International Joint Conference on
Print_ISBN :
0-7803-8359-1
DOI :
10.1109/IJCNN.2004.1380078