On the adaptive stabilization and ergodic behaviour of stochastic jump-Markov systems via nonlinear filtering

The authors propose an adaptive control method for a continuous-time linear stochastic system with unobserved finite-state jump-Markov parameters (parameters constituting a Markov process evolving on a finite set), also called a linear hybrid system. It is assumed that the system is time-independent and that its states are completely observed. By applying the optimal nonlinear filter, the parameters are estimated based on observations of the output. A class of adaptive state feedback algorithms, dependent on the nonlinear filter output, is proposed, and a Lyapunov function argument shows that under certain conditions, for any finite initial probability distribution, the resulting system is stochastically stable. In addition, it is proved that, with any (stochastically) stabilizing adaptive state feedback, the system is weakly controllable (accessible) for any initial condition. Stochastic stability, for any homogeneous diffusion process, implies that there exists an invariant probability distribution for the process, unique with respect to the initial condition. Moreover, it is proved that weak controllability results in the ergodicity of the process for every initial condition