Estimation entropy and its operational characteristics in information acquisition systems

We consider a pair of correlated processes in {Z_n}_n=-infin ^infin and {Z_n}_n=-infin ^infin, where the former is observable and the latter is hidden. The uncertainty in the estimation of S_n upon the finite past history of Z₀ ^n-infin1 is H(S_n|)Z₀ ^ninfin1 which is a sequence of n. The limit of Cesaro mean of this sequence is called the estimation entropy. We show that the estimation entropy is the long run average entropy of the belief state on the hidden process obtained from the observation process. Estimation entropy inversely measures the observability of the hidden process through the observed process, and its minimization is the goal for optimal observability problems such as sensor scheduling. In this paper we describe such an operational characterization of estimation entropy.