New training algorithms for dependency initialized multilayer perceptrons

Due to the chaotic nature of multilayer perceptron training, training error usually fails to be a monotonically nonincreasing function of the number of hidden units. New training algorithms are developed where weights and thresholds from a well-trained smaller network are used to initialize a larger network. Methods are also developed to reduce the total amount of training required. It is shown that this technique yields an error curve that is a monotonic nonincreasing function of the number of hidden units and significantly reduces the training complexity. Additional results are presented based on using different probability distributions to generate the initial weights.