A comparative study on GA based and BP based induction of neural network trees

Neural network tree (NNTree) is a special multivariate decision tree (DT) with each nonterminal node containing an expert neural network (ENN). Generally speaking, NNTrees can outperform standard DTs because the ENNs can extract more complex features. However, induction of multivariate DTs is very difficult. Even if each nonterminal node contains a simple oblique hyperplane, the induction problem can be NP-complete. To solve this problem, we have introduced an evolutionary algorithm that follows the same recursive procedure for inducing a standard DT, and designs an ENN for each nonterminal node using GA (genetic algorithm). This algorithm, however, is very time consuming and cannot be used easily. In this paper, we propose two new methods. One is to evolve the whole tree instead of evolving the ENNs recursively. Another is to define the group labels for the examples assigned to each nonterminal node using a heuristic method, and design the ENNs with the back propagation (BP) algorithm. Experimental results with 10 public databases show that the BP based algorithm is much more efficient than GA based algorithms.