A Natural Algorithmic Approach to the Structural Optimisation of Neural Networks

Structural design of an artificial neural network is a very important phase in the construction of such a network. The selection of the optimal number of hidden layers and hidden nodes has a significant contribution to the performance of a neural network, though it is typically decided in an adhoc manner. In this paper, the structure of a neural network is adaptively optimised by determining the number of hidden layers and hidden nodes that give the optimal performance in a given problem domain. An optimisation approach was developed based on the particle swarm optimisation (PSO) algorithm, which is a simple, easy to implement but highly effective evolutionary algorithm which uses a cooperative approach. This approach adaptively optimises an artificial neural network built for a specific problem domain by evolving both the hidden layers and the number of nodes in a particular hidden layer. It has been applied on two well known case studies in the classification domain, namely the Iris data classification and the ionosphere data classification. The obtained results and comparisons done with past research work has clearly shown that this method of optimisation is by far, the best approach for adaptive structural optimisation of artificial neural networks.