Uncertainty bounds in system identification with limited data

Consider the problem of determining uncertainty bounds for parameter estimates in system identification. Calculating uncertainty bounds requires information about the distribution of the estimate. Although many common parameter estimation methods (e.g., maximum likelihood, least squares, maximum a posteriori, etc.) have an asymptotic normal distribution, very little is usually known about the finite-sample distribution, even when the underlying models are linear. This paper presents a method for characterizing the distribution of an estimate when the sample size is small. The approach works by comparing the actual (unknown) distribution of the estimate with an “idealized” (known) distribution. Some discussion and analysis are included that compare the approach here with the well-known bootstrap and saddlepoint methods from statistics. Example applications of the approach are presented in the areas of signal-plus-noise modeling, nonlinear regression, and time series correlation analysis