Greedy Adaptive Linear Compression in Signal-Plus-Noise Models

In this paper, we examine adaptive compression policies, when the sequence of vector-valued measurements to be compressed is noisy and the compressed variables are themselves noisy. The optimization criterion is information gain. In the case of sequential scalar compressions, the unit-norm compression vectors that greedily maximize per-stage information gain are eigenvectors of an a priori error covariance matrix, and the greedy policy selects them according to eigenvalues of a posterior covariance matrix. These eigenvalues depend on all previous compressions and are computed recursively. A water-filling solution is given for the optimum compression policy that maximizes net information gain, under a constraint on the average norm of compression vectors. We provide sufficient conditions under which the greedy policy for maximizing stepwise information gain actually is optimal in the sense of maximizing the net information gain. In the case of scalar compressions, our examples and simulation results illustrate that the greedy policy can be quite close to optimal when the noise sequences are white.