Stochastic approximation for consensus over general digraphs with Markovian switches

This paper considers consensus problems with Markovian switching networks and noisy measurements, and stochastic approximation is used to achieve mean square consensus. The main contribution of this paper is to obtain ergodicity results for backward products of degenerating stochastic matrices with Markovian switches, and subsequently prove mean square consensus for the stochastic approximation algorithm. Our ergodicity proof is to build a higher dimensional dynamical system and exploit its two-scale feature.