A Stochastic Approximation Technique for Generating Maximum Likelihood Parameter Estimates

This paper shows how stochastic approximation (SA) can be used to construct maximum likelihood estimates of system parameters. The procedure described here relies on a derivative approximation other than the usual finite-difference approximation associated with a Kiefer-Wolfowitz SA procedure. This alternative derivative approximation requires fewer, by a factor equal to the dimension of the parameter vector being estimated, computations than the standard finite-difference approximation. Numerical evidence presented in the paper indicates that this SA procedure is, relative to a Kiefer-Wolfowitz procedure, most efficient when considering large-scale systems.