On asymptotic properties of supervised learning

Summary form only given, as follows. Certain asymptotic properties of supervised learning algorithms are developed. It is noted that there is a strong resemblance between the supervised learning and adaptive filtering problems. Almost sure convergence is established. The usual IID assumption on the data is weakened. The limiting distribution of a normalized sequence is obtained. In fact, a functional central limit theorem is derived. Such asymptotic normality is then utilized to design a practical stopping rule, which is based on the construction of ellipsoidal confidence regions. To ensure the boundedness of the iterates in the learning algorithms, several truncation or projection methods are described and the convergence is discussed.<>