Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes

Author

Tadic, Vladislav B. ; Meyn, Sean P.

Author_Institution

Dept. of Electr. & Electron. Eng., Melbourne Univ., Parkville, Vic., Australia

Volume

5

fYear

2003

fDate

4-6 June 2003

Firstpage

4426

Abstract

Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes are analyzed in this paper. The analysis is carried out for the algorithms with additive noise, as well as for the algorithms with non-additive noise. The algorithms with additive noise are considered for the case where the noise id state-dependent and admits the decomposition as a sum of a martingale difference sequence and a telescoping sequence. The algorithms with non-additive noise are analyzed for the case where the noise satisfies uniform or strong mixing conditions, as well as for the case where the noise is a Markov chain controlled by the algorithm states.

Keywords

Markov processes; approximation theory; noise; sequences; Markov chain; additive noise; algorithm states; asymptotic properties; constant step sizes; martingale difference sequence; nonadditive noise; state dependent noise; telescoping sequence; two time scale stochastic approximation algorithms; Additive noise; Algorithm design and analysis; Approximation algorithms; Communication networks; Machine learning; Machine learning algorithms; Signal processing algorithms; Size control; Stochastic processes; Stochastic resonance;

fLanguage

English

Publisher

ieee

Conference_Titel

American Control Conference, 2003. Proceedings of the 2003

ISSN

0743-1619

Print_ISBN

0-7803-7896-2

Type

conf

DOI

10.1109/ACC.2003.1240536

Filename

1240536