Controlled sensing for multihypothesis testing based on Markovian observations

A new model for controlled sensing for multiphypothesis testing is proposed and studied in both the sequential and fixed sample size settings. This new model, termed a stationary Markov model, exhibits a more complicated memory structure in the controlled observations than the existing stationary memoryless model. In the sequential setting, an asymptotically optimal sequential test using a stationary causal Markov control policy enjoying a strong asymptotic optimality condition is proposed for this new model, and its asymptotic performance is characterized. In the fixed sample size setting, bounds for the optimal error exponent for binary hypothesis testing are derived; it is conjectured that the structure of the asymptotically optimal control for the stationary Markov model will be much more complicated than that for the stationary memoryless model.