An Improved Online quasi-Newton method for robust training and its application to microwave neural network models

This paper describes a new technique for robust training of feedforward neural networks. The proposed algorithm is employed for the robust neural network training purpose. The quasi-Newton method was studied as one of the most efficient optimization algorithms based on the gradient descent and used as the batch training method of neural networks. On the other hand, the stochastic (online) quasi-Newton method was developed as an algorithm for the machine learning. In this paper the stochastic quasi-Newton training algorithm is improved for robust neural network training. Neural network training for some benchmark problems is presented to demonstrate the proposed algorithm. Furthermore, neural network training for microwave circuit modeling, such as the waveguide and the microstrip examples is presented, demonstrating that the proposed algorithm achieves more accurate models than both the batch and the stochastic quasi-Newton methods.