Architecture selection for neural networks

Researchers worldwide converge into the fact that the exhaustive search over the space of network architectures is computationally infeasible even for networks of modest size. The use of heuristic strategies that dramatically reduce the search complexity is a common technique. These heuristic approaches employ directed search algorithms, such as selection of the number of nodes via sequential network construction (SNC), pruning inputs and weights via sensitivity based pruning (SBP) and optimal brain damage (OBD). The main disadvantage of the particular techniques is the production of one hidden layer only perceptrons and not a multiplayer network. As a consequence we cannot ensure that the resulted network will produce "optimal" results. This paper investigates the use of other heuristic strategies that produce multiplayer perceptrons. We propose three algorithms for the construction of networks and compare them with the classic approaches. Simulation results show that these methods lead to "near-optimal" network structures by engaging error-related information