On the role of mismatch in rate distortion theory

Using a codebook C, a source sequence is described by the codeword that is closest to it according to the distortion measure d₀(x,xˆ₀). Based on this description, the source sequence is reconstructed to minimize the reconstruction distortion as measured by d₁(x,xˆ₁), where, in general, d₁(x,xˆ₁)≠d₀(x,xˆ₀ ). We study the minimum resulting d₁(x,xˆ₁ )-distortion between the reconstructed sequence and the source sequence as we optimize over the codebook subject to a rate constraint. Using a random coding argument we derive an upper bound on the resulting distortion. Applying this bound to blocks of source symbols we construct a sequence of bounds which are shown to converge to the least distortion achievable in this setup. This solves the rate distortion dual of an open problem related to the capacity of channels with a given decoding rule-the mismatch capacity. Addressing a different kind of mismatch, we also study the mean-squared error description of non-Gaussian sources with random Gaussian codebooks. It is shown that the use of a Gaussian codebook to compress any ergodic source results in an average distortion which depends on the source via its second moment only. The source with a given second moment that is most difficult to describe is the memoryless zero-mean Gaussian source, and it is best described using a Gaussian codebook. Once a Gaussian codebook is used, we show that all sources of a given second moment become equally hard to describe