Title :
A novel neural network training technique based on a multi-algorithm constrained optimization strategy
Author :
Karras, Dimitris A. ; Lagaris, Isaak E.
Author_Institution :
Dept. of Inf., Ioannina Univ., Greece
Abstract :
A novel methodology for efficient offline training of multilayer perceptrons (MLPs) is presented. The training is formulated as an optimization problem subject to box-constraints for the weights, so as to enhance the network´s generalization capability. An optimization strategy is used combining variable metric, conjugate gradient and no-derivative pattern search methods that renders the training process robust and efficient. The superiority of this approach, over Off-line Backpropagation algorithm, the RPROP training procedure as well as over the stand alone algorithms involved in the proposed complex optimization strategy, is demonstrated by direct application to two real world benchmarks and the parity-4 problem. These problems have been obtained from a standard collection of such benchmarks and special care has been taken on the statistical significance of the results by organizing the experimental study so as to compare the averages and variances of the training and generalization performance of the algorithms involved
Keywords :
constraint handling; learning (artificial intelligence); multilayer perceptrons; box-constraints; conjugate gradient; multi-algorithm constrained optimization strategy; multilayer perceptrons; network´s generalization capability; neural network training technique; offline training; optimization problem; parity-4 problem; pattern search methods; real world benchmarks; variable metric; Backpropagation algorithms; Constraint optimization; Informatics; Multilayer perceptrons; Neural networks; Optimization methods; Organizing; Robustness; Search methods; Weight control;
Conference_Titel :
Euromicro Conference, 1998. Proceedings. 24th
Conference_Location :
Vasteras
Print_ISBN :
0-8186-8646-4
DOI :
10.1109/EURMIC.1998.708088