Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity

This paper has three main goals: (i) to employ two classes of algorithms: bio-inspired and gradient-based to train multi-layer perceptron (MLP) neural networks for pattern classification; (ii) to combine the trained neural networks into ensembles of classifiers; and (iii) to investigate the influence of diversity in the classification performance of individual and ensembles of classifiers. The optimization version of an artificial immune network, named opt-aiNet, particle swarm optimization (PSO) and an evolutionary algorithm (EA) are used as bio-inspired methods to train MLP networks. Besides, the standard backpropagation with momentum (BPM), a quasi-Newton method called DFP and a modified scaled-conjugate gradient (SCGM) are the gradient-based algorithms used to train MLP networks in this work. Comparisons among all the training methods are presented in terms of classification accuracy and diversity of the solutions found. The results obtained suggest that most bio-inspired algorithms deteriorate the diversity of solutions during the search, while immune-based methods, like opt-aiNet, and multiple initializations of standard gradient-based algorithms provide diverse solutions that result in good classification accuracy for the ensembles.