On an efficient design algorithm for modular neural networks

A multilayer perceptron has been known to have the capability of approximating any smooth functions to any desired accuracy if it has a sufficient number of hidden neurons. But its training, based on the gradient method, is usually a time consuming procedure that may converge toward a poor local minimum. In this paper, the authors first propose a constructive design method (CDM) for two-layer perceptrons based on the analysis of a given data set. Here, some feature vectors, which can characterize the function “well” are extracted and used to construct a two-layer perceptron. But when the classes of the feature vectors are not linearly separable, the network may not approximate the function well, mainly due to the interference among the hyperplanes of hidden neurons. Second, to compensate for this interference, a systematic design procedure for the construction of modular neural networks (MNN) has been proposed. In this procedure, after partitioning the input space into many local regions a two-layer perceptron constructed by the CDM is assigned to each local region. By doing this, the classes in each local region may become linearly separable and as a result, the function may be approximated to any desired accuracy by the MNN