An efficient training algorithm for multilayer neural networks by homotopy continuation method

In this paper, the training of multilayer neural networks is expressed as the problem of solving a system of nonlinear equations. The weights in the network are considered as the variables of the nonlinear equations. Moreover, the nonlinear equations can be solved by using homotopy-based continuation methods after the entire training data are presented to the network. Unlike gradient-based algorithm, it can almost be constructed to be globally convergent. The experimental results on both the parity checker and encoder/decoder problem show the excellent convergence behavior of homotopy continuation method in contrast with backpropagation algorithm