Rates of convergence for an adaptive filtering algorithm

The strong invariance principle of probability theory asserts that the running sum of a given sequence of independent and identically distributed zero-mean second-order random variables (of any underlying distribution) can be approximated almost surely (a.s.) by the sample paths of a suitable Brownian motion. This has many ramifications, because it allows us to use very detailed understanding of the sample properties of Brownian motion to obtain a precise characterization of the asymptotic behaviour of the given sum of random variables. Motivated by the invariance principle, our goal is to obtain an a.s. approximation of the "difference" process [h_i - h_*] by some "standard" process whose sample-path properties are well understood. We shall show that the difference process can in fact be approximated a.s. by a Gauss-Markov process which is itself a linear function of a suitable Brownian motion. To illustrate the applicability of this result, we use it to prove a law of the iterated logarithm (which in turn implies a precise and unimprovable a.s. rate of convergence of h _i to h_*) and a functional central limit theorem for [h_i - h_*]