A new pyramid network and its generalization performance

A new pyramid network with novel topology is proposed, based on functional expansion over the basis of the output functions of all hidden layer units. In particular, its output has connections to the outputs of all hidden layer units of a multilayer network. Then supervised training rules for weight coefficients of the pyramid network are shown so that these functional basis are generated recursively at each hidden layers and by training of synapse coefficients. The proposed networks show superior generalization capability in simulations with handwritten figures recognition, especially in cases of small training data. Besides, their performances to alleviate local minima problem and effective trimming of the network are also investigated