Adaptive estimation of HMM transition probabilities

This paper presents new schemes for recursive estimation of the state transition probabilities for hidden Markov models (HMMs) via recursive prediction error (RPE) methods. These new schemes are designed to be consistent and well conditioned, compared to the previous RPE schemes which are known to be ill-conditioned in low noise environments. The RPE algorithms proposed in this paper, although requiring less computational effort than the previous algorithms are still of order N ⁴, each time instant, where N is the number of Markov states. Extended least squares algorithms are also presented which less computational effort (order N² per time instant) but for which no convergence results are presented. A consistent algorithm for simultaneous estimation of the state output levels and the state transition probabilities is also presented and discussed. Implementation aspects of all proposed algorithms are discussed, and simulation studies are presented to illustrate convergence and convergence rates