Incremental least squares methods and the extended Kalman filter

Proposes and analyzes nonlinear least squares methods, which process the data incrementally, one data block at a time. Such methods are well suited for large data sets and real time operation, and have received much attention in the context of neural network training problems. The author focuses on the extended Kalman filter, which may be viewed as an incremental version of the Gauss-Newton method. The author provides a nonstochastic analysis of its convergence properties, and discusses variants aimed at accelerating its convergence