Interactive Encoding and Decoding for One Way Learning: Near Lossless Recovery With Side Information at the Decoder

A source coding paradigm called interactive encoding and decoding (IED) is considered for a source network where a finite alphabet source X is to be encoded, and another finite alphabet source Y correlated with X is available only to the decoder as a helper. The optimal performance achievable asymptotically (OPAA) by IED is investigated, where the performance is measured as the average number of bits per symbol exchanged by the encoder and decoder until the decoder learns X with high probability. First, it is shown that for any stationary (X, Y), the OPAA by IED is given by the conditional entropy rate H(X|Y) of X given Y. This is in contrast with noninteractive Slepian-Wolf (SW) coding, where the OPAA is shown in general to be strictly greater than H(X|Y) when (X, Y) is not ergodic. Second, for a memoryless source pair (X, Y), it is shown that IED approaches H(X|Y) faster than SW coding does. Finally, it is demonstrated that one can convert any classical universal data compression algorithm with side information to a universal IED algorithm for the class ¿ of all stationary ergodic source pairs. In contrast, universal SW coding algorithms for the class ¿ do not exist.