The value of sleeping: A rollout algorithm for sensor scheduling in HMMs

This paper presents a new Q-value approximation algorithm for joint sensor scheduling and MAP state estimation in hidden Markov models. The proposed algorithm is motivated by the fact that energy-constrained embedded devices spend a significant amount of time in sleep modes. To develop an adaptive sensing-resource scheduling policy, the proposed base policy computes the exact value of sleeping over an infinite time horizon. This value is incorporated to rank sensing resources, trading off sensing quality with usage cost. As the base policy is independent of the sensing modalities, the proposed method is useful in applications where observation parameters such as SNR are time-varying, and when re-optimization is not practical. For applications with significant energy constraints, the proposed policy performs better than other heuristics and achieves near optimal performance/resource trade-off, as demonstrated in a long-term energy-constrained wildlife monitoring application.