Neural network training using homotopy continuation methods

Neural networks are widely used in performing classification tasks. The networks are traditionally trained using gradient methods to minimize the training error. These techniques, however, are highly susceptible to getting trapped in local minima. The authors propose an innovative approach to obtain the global minimum of the training error. The globally optimum solution can be obtained by employing the homotopy continuation method for minimizing the classification error during training. Two different approaches are considered. The first approach involves the polynomial modeling of the nodal activation function and the second approach involves the traditional sigmoid function. Results illustrating the superiority of the homotopy method over the gradient descent method are presented