Acceleration of learning speed in neural networks by reducing weight oscillations

We propose a novel fast learning algorithm in neural networks. The conventional backpropagation algorithm suffers from slow convergence due to weight oscillations at a narrow valley in the error surface. To overcome this difficulty we derive a new gradient term by modifying the original gradient term with an estimated downward direction at a valley. Simulation results show that the proposed method reduces oscillations considerably and achieves fast convergence