Monte Carlo-based computation of the Fisher information matrix in nonstandard settings

The Fisher information matrix plays a central role in the practice and theory of estimation. This matrix provides a summary of the amount of information in the data relative to the quantities of interest. There are many applications of the information matrix in modeling and system identification, including input design, confidence region calculation, and prediction bounds. This paper summarizes some basic principles associated with the information matrix, presents a resampling-based method for computing the information matrix together with some new theory related to efficient implementation, and presents some numerical studies. The resampling-based method relies on an efficient technique for estimating the Hessian matrix, introduced as part of the adaptive ("second-order") form of the simultaneous perturbation stochastic approximation (SPSA) optimization algorithm.