A new dynamic optimal learning rate for a two-layer neural network

The learning rate is crucial for the training process of a two-layer neural network (NN). Therefore, many researches have been done to find the optimal learning rate so that maximum error reduction can be achieved in all iterations. However, in this paper, we found that the best learning rate can be further improved. In saying so, we have revised the direction to search for a new dynamic optimal learning rate, which can have a better convergence in less iteration count than previous approach. There exists a ratio k between out new optimal learning rate and the previous one after the first iteration. In contrast to earlier approaches, the new optimal learning rate of the two-layer NN has a better performance in the same experiment. So we can conclude that our new dynamic optimal learning rate can be a very useful one for the applications of neural networks.