Issues in benchmarking of ANN training algorithms

There is a need for a consistent and effective method to evaluate algorithms for various aspects of training feedforward networks, such as weight initialization, training data selection, error minimization, and weight decay/pruning. We feel that this should be addressed by the construction and application of a benchmark, that is, a comprehensive set of training problems and evaluation criteria. This paper discusses a number of issues which must be addressed in the formation of such a benchmark. Firstly, a taxonomy of learning problems must be derived. This involves issues such as the nature of the mapping, the nature of the training data, and the learning criteria. Secondly, training algorithm performance criteria must be established; these may be dependent upon the class of learning problem. Thirdly, a common software framework for evaluation of training algorithm modules must be designed. Finally, a benchmark set of learning problems must be developed for evaluation of the range of training-related algorithms, as applied to the range of learning problems. Early experiences in benchmarking ANN training algorithms are presented