Some classical constructive neural networks and their new developments

Reviewing old ones is to better understand new ones and also for innovating. The mapping capability of artificial neural networks is dependent on their structure, i.e., the number of layers and the number of hidden units. Presently, there is no formal way of computing network topology as a function of the complexity of a problem; it is usually selected by trial-and-error and can be rather time consuming. Basically, we make use of two mechanisms that may modify the topology of the network: growth and pruning. This paper firstly discusses some learning algorithms and topologies of classical constructive neural networks. Only incremental or growing algorithms employing supervised learning algorithms are outlined here which includes Tiling algorithm, Tower algorithm, Upstart algorithm, Cascade-Correlation algorithm, Restricted coulomb energy network and Resource-allocation network. For each neural network model, we review their topology structure and learning features. The new development of constructive neural networks is given at the end of the paper.