Memetic Self-Configuring Genetic Programming for Solving Machine Learning Problems

A hybridization of self-configuring genetic programming algorithms (SelfCGPs) with a local search in the space of trees is fulfilled to improve their performance for symbolic regression problem solving and artificial neural network automated design. The local search is implemented with two neighborhood systems (1-level and 2-level neighborhoods), three strategies of a tree scanning ("full", "incomplete" and "truncated") and two ways of a movement between adjacent trees (transition by the first improvement and the steepest descent). The Lamarckian local search is applied on each generation to ten percent of best individuals. The performance of all developed memetic algorithms is estimated on a representative set of test problems of the functions approximation as well as on real-world machine learning problems. It is shown that developed memetic algorithms require comparable amount of computational efforts but outperform the original SelfCGPs both for the symbolic regression and neural network design. The best variant of the local search always uses the steepest descent but different tree scanning strategies, namely, full scanning for the solving of symbolic regression problems and incomplete scanning for the neural network automated design. Additional advantage of the approach proposed is a possibility of the automated features selection.