Effect of a sparse architecture on generalization behaviour of connectionist networks: a comparative study

Generalization, the ability of achieving equal performance with respect to the training patterns for the design of the system as well as the unknown test patterns outside the training set, is considered to be the most desirable aspect of a cognitive learning system. For connectionist classifiers, generalization depends on several factors like the network architecture and size, learning algorithm, complexity of the problem and the quality and quantity of the training samples. Various studies on the generalization behaviour of a neural classifier suggest that the architecture and the size of the network should match the size and complexity of the training sample set of the particular problem. The popular ideas for improving generalization is network pruning by removing redundant nodes or growing by adding nodes to reach the optimum size for matching. In this work a feedforward multilayer connectionist model proposed earlier by Chakraborty et al. (1997) with a sparse fractal connection structure between the neurons of adjacent layers has been studied for its suitability compared to other architectures for achieving good generalization. A comparative study with another feedforward multilayer model with network pruning algorithms for pattern classification problem revealed that it is easier to achieve good generalization with the proposed sparse fractal architecture