Weight evolution algorithm with dynamic offset range

The main problems for gradient-descent algorithms such as backpropagation are its slow convergence rate and the possibility of being trapped in local minima. In this paper, a weight evolution algorithm with dynamic offset range is proposed to remedy the above problems. The idea of weight evolution is to evolve the network weights in a controlled manner during the learning phase of backpropagation so as to jump to the regions of smaller mean squared error whenever the backpropagation stops at a local minimum. If the algorithm is consistently being trapped in a local minimum, the offset range for weight evolution will be incremented to allow larger weight space to be searched. When the local minimum is bypassed, the offset range will be reset to the initial value. It can be proved that this method can always escape local minima and guarantee convergence to the global solution. Simulation results show that the weight evolution algorithm with dynamic offset range gives a faster convergence rate and global search capability