Encoder and decoder optimization for source-channel prediction in error resilient video transmission

Motion-compensated prediction that accounts for loss in the channel is achieved by the source-channel prediction (SCP) method, which is based on the expected decoder reconstruction of past frames (rather than their encoder reconstruction). The decoder reconstruction is estimated by exploiting the recursive optimal per-pixel estimate (ROPE), which explicitly accounts for the quantization distortion, channel loss, error propagation, as well as the decoder operation and achieves improved error resilience. We take this paradigm further by noting that the decoder can, in turn, be reoptimized to match the modification introduced to the encoder for SCP. Simulation results demonstrate substantial performance gains over conventional decoding. We then examine the benefits of re-optimizing the encoder for the newly matched decoder, and then reoptimizing the decoder, etc. and note that further incremental gains are minor. Hence, one complete round of SCP optimization offers significant gains, but multiple reoptimization iterations may not be cost-effective.