An information theoretic design and training algorithm for neural networks

An algorithmic approach to designing feedforward neural networks for pattern classification is presented. The technique computes a single hidden layer of nodes by adding one node at a time until the desired classification has been achieved. At each iteration, a node that maximizes an information theoretic measure is selected from a collection of candidates. The methodology is heuristic in nature, intending to solve the NP-hard problem of constructing a neural network with a minimum number of nodes. Two strategies for computing a collection of candidate nodes are presented and some experimental results obtained by using the strategies are reported