RGB and depth intra-frame Cross-Compression for low bandwidth 3D video

With the recent explosion in the development of multimedia hardware capable of 3D display, 3D Picture Coding Systems have assumed a pivotal role. While encoding techniques for stereo-scopic images is a well researched topic and compression standards such as MPEG provide variants to support it, compression of RGB-D data such as from the Microsoft Kinect sensor offers a number of unsolved challenges. Projected texture based active sensors such as the Kinect offer a number of advantages in comparison with traditional 3D capture systems. While not affecting the visible spectrum of the scene these sensors are capable of producing highly accurate 3D reconstructions of complex scenes (as well as novel viewpoints) - even those with homogeneous surfaces lacking textural features that form the foundation of stereoscopic range measurement systems. Conventional approaches to compressing the RGB and D images separately are suboptimal in terms of compression efficiency, bandwidth usage and scalability. On the other hand, state-of-the-art methods in the field are not suitable for low bandwidth applications, typical of mobile phone devices, on-field civilian and defense robotic systems, especially those operating on unreliable or high-loss wireless networks. In order to address these concerns, we present a novel RGB-D Cross-Compression algorithm that can be used for static 3D scene reconstruction as well as intra-frame coding of 3D videos. The algorithm detects salient edge-like structures in RGB and D images and perform cross-coding across the modalities to yield a scalable system for 3D video coding. Results presented using the Microsoft Ballet and Breakdancers test sequences demonstrate the efficiency of the system in terms of compression rate, reconstruction quality and rate-distortion characteristics. The scalability of the approach also makes it well suited for mobile and wireless applications.