Effects of initialization on structure formation and generalization of neural networks

In this paper, we propose an initialization method of multilayer neural networks (NN) employing the structure learning with forgetting. The proposed initialization consists of two steps: weights of hidden units are initialized so that their hyperplanes should pass through the center of input pattern set, and those of output units are initialized to zero. Several simulations were performed to study how the initialization affects the structure forming process of the NN. From the simulation result, it was confirmed that the initialization gives better network structure and higher generalization ability