A new algorithm for training multilayer feedforward neural networks

The authors present a new learning and synthesis algorithm for training multilayer feedforward neural networks. Its principle is to synthesize a neural network by growing layers based on using training results until the required results are achieved. Each layer is trained with the pocket algorithm and hidden neurons are added only when needed. The proposed algorithm has the following properties. 1) The architecture of the network is generated dynamically by the learning process algorithm and it is unnecessary to estimate the number of layers and the number of hidden neurons before training. The neuron activation function is hard limiting instead of sigmoidal. 2) The learning speed is faster than other algorithms, especially the backpropagation algorithm. After the neural network is fully trained the system error is absolutely zero. 3) This algorithm can classify both linear separable and linear nonseparable families, whereas the backpropagation algorithm will fail sometimes. Extensive numerical simulation studies of this algorithm have confirmed these properties and thus the proposed training strategy looks promising