Towards constructing optimal feedforward neural networks with learning and generalization capabilities

The authors consider the problem of finding minimal neural networks (in terms of number of neurons and synapses) subject to desired learning and generalization capabilities. An algorithm which automatically determines the number of neurons and the location of synaptic connections is proposed. A new neural network model is introduced to facilitate solving the optimal architecture problem. The synaptic connections are pruned based on testing hypotheses that the corresponding weights be smaller than cutting thresholds. Simulation results are demonstrated for designing neural networks for: (1) a 7-segment electronic display; and (2) a power system load modeling problem. Optimal architecture (in the sense of achieving the lower bound on the number of neurons) are obtained for (1), and a 50%-60% save-up of synapses with the desired learning/generalization capabilities is obtained for (2)