Measurement of a Markov jump process by hypothesis tests

This paper considers measurement of a continuous-time finite-state Markov jump process. The measurements take the form of hypothesis tests, which determine whether the system is in an inspected state or not. This model is relevant to determining policies for performing medical exams, inspecting machinery, and tracking targets. Measurement policies are selected so as to minimize an average cost criterion that penalizes both uncertainty about the state of the system and the frequency of measurements. Unlike observation control for LQG systems, where the optimal input to a Kalman filter may be selected by an open-loop policy, for Markov jump processes, the optimal measurement schedule depends on the outcomes of prior measurements. The policy iteration algorithm is shown to give a convergent method for finding optimal feedback measurement rules. The model is applied to find the optimal screening frequency for diabetes.