Optimizing Distributed Source Coding for Interactive Multiview Video Streaming Over Lossy Networks

In interactive multiview video streaming (IMVS), a user observes one view at a time, but can periodically switch to a desired neighboring captured view as the video is played back in time. Previous IMVS works focus on efficient compression techniques that facilitate interactive view switching. In this paper, in addition to the loss-resilient aspect during network streaming we address how to design efficient coding tools and optimize frame structure for transmission to facilitate view switching and contain error propagation in differentially coded video due to packet losses. We first design a new unified distributed source coding (uDSC) frame-a new coding tool that simultaneously offers view switching and loss-resilient capabilities-for periodic insertion into the multiview frame structure. After inserting uDSC-frames into the coding structure, we schedule packets for network transmission in a rate-distortion optimal manner for both wireless multicast and wired unicast streaming scenarios. For wireless multicast over a Gilbert-Elliott loss model, frames in a group of pictures are packetized and reordered, so that uDSC frames are correctly decoded with high probability, mitigating error propagation. For wired unicast, we use a Markov decision process to optimize packet transmission to minimize expected distortion given a bandwidth constraint. Experimental results show that systems that insert uDSC frames and optimize packet transmission can outperform other competing coding schemes by up to 2.8 and 11.6 dB in wireless multicast and wired unicast streaming scenarios, respectively.