Convergence properties of continuous-time Markov chains with application to target search

This paper considers the search for targets modeled as a discrete state, continuous-time Markov process. Convergence properties are analyzed using the eigenvalues and eigenvectors of a state transition rate matrix without explicitly solving differential equations or calculating matrix exponentials. It also studies the effect of cueing on convergence rate using eigenvalue analysis and optimal control theoretic approach.