On the properties of nonlinear POMDPs for active state tracking

The problem of active state tracking of a discrete-time, finite-state Markov chain is considered. To enhance state estimates, noisy observations are dynamically collected by exerting appropriate control over their information content. In our earlier work, a Kalman-like estimator was proposed and the optimal observation selection policy was derived via dynamic programming. The resulting partially observable Markov decision process proves to be nonlinear, thus challenging the design of the control policy. Herein, the focus is on deriving cost-to-go function´ properties and characterizing sufficient conditions regarding the structure of the optimal control policy.