Combining Gradient and Evolutionary Approaches to the Artificial Neural Networks Training According to Principles of Support Vector Machines

A gradient based learning for ANN training in pattern recognition tasks and a genetic approach for ANN pruning are proposed in this paper. The goal is to achieve a wide margin classifier the Vapnik-Chevornenkis (VC) dimension of which is being reduced in order to increase the generalization performance. Inspired by Support Vector Machines the examples closest to the decision boundary contribute to the training the most. The training penalty is rule-based and calculated according to the spatial distribution of the training examples relative to the separating hyperplane. The tendency to saturation of hidden neurons is suppressed. Genetic algorithm based method is proposed for reduction of the size of a trained ANN. The proposed algorithms were tested on artificial and real world data and compared to standard Backpropagation and Support Vector Machine with Gaussian RBF kernel.