The properties of Mth order differences and their relationship to Mth order random stability

Allan and Hadamard variances are well-known examples of M^th order difference (Δ) variances. These are mean square (MS) averages over data of the M^th order Δ-measure Δ(τ)^Mv(t), where Δ(τ)v(t) = v(t+τ) - v(t). Many of the important properties of Allan and Hadamard variances are specific examples of the more general properties of M^th order Δ-measures. This paper first provides a useful compendium of these properties along with proofs. Allan and Hadamard variances are also known as MS measures of M^th order random stability (or instability). But what is M^th order random stability? The second part of this paper offers a definition of such stability as the prediction error solely due to the random error or noise in the data at an (M+1)^th point from an M-parameter calibration function that is passed through M previously measured data points. Using this definition and the properties of Δ-measures, it is shown that M^th order Δ-variances are natural measures of M^th order MS random stability when the deterministic drift in the data has (M-1)^th or lower order polynomial behavior, the calibration function is an (M-1)^th polynomial, and the measured data and predicted value are all spaced by τ. When the drift (aging plus environmental variation) does not have this behavior, it is further shown that Δ-variances are biased measures of such polynomial random stability, even when one uses fitting techniques to remove the drift from the data. This drift removal process is shown to alter the spectral kernel K_stat(f) that relates the Δ-variance in question to the power spectral density (PSD) of the random noise in the data, even if the fit perfectly removes the direct drift effects. Methods are presented for computing such altered K_stat(f) and ar- - e illustrated by computing the changes in K_stat(f) that occur due to various drift-removal techniques in the unmodified Allan variance. These results are then used to explain why biases in drift-removed Δ-variances can change so drastically as a function of power law PSD type and fit methodology. Finally, it is shown that M^th order Δ-measures that are natural measures of random stability in the presence of intrinsic aging can become highly biased as a result of environmental drift removal.