Application of an Adaptive Differential Evolution Algorithm With Multiple Trial Vectors to Artificial Neural Network Training

In this paper, an application of an adaptive differential evolution (DE) algorithm with multiple trial vectors for training artificial neural networks (ANNs) is presented. The proposed method is DE-ANNT+, which is a DE-ANN Training (DE-ANNT) modified by adding a multiple trial vectors technique. DE-ANNT+ allows one to train an ANN of arbitrary architectures, and it offers a nondifferentiable neuron activation function. In contrast to a basic DE algorithm, DE-ANNT+ possesses two modifications. In DE-ANNT+, adaptive selection of control parameters and a multiple trial vectors technique are introduced. Adaptive selection means that the number of required parameters of the algorithm is decreased. The multiple trial vectors technique increases the probability of generating a better solution because a greater number of temporary solutions is generated around the existing solutions. The DE-ANNT+ algorithm, with these two modifications, is used for ANN training to classify the parity-p problem. The results from the obtained algorithm have been compared with results from the following algorithms: an evolutionary algorithm, a DE algorithm without multiple trial vectors, gradient training methods, such as error back-propagation, and the Levenberg-Marquardt method.