MPEG-4-based 3D video contents creation method using time-of-flight sensor for 3DTV

Three-dimensional (3D) TV is considered as the next-generation broadcasting service. In this paper, we propose a new scheme to generate 3D video contents using a time-of-flight (TOF) range sensor for the 3DTV service. TOF sensors are a relatively novel technology allowing for the real-time capture of both photometric and geometric scene information. In this work, in order to generate natural 3D video, we develop a practical pipeline including TOF data processing and MPEG- 4-based data transmission and reception. First, we acquire color and depth videos from a TOF range sensor. Then, alpha matting and depth map enhancement are performed to handle fuzzy and hairy objects that are traditionally difficult in image processing. Thereafter, the generated video-plus-depth data are combined with interactive computer graphic models in the MPEG-4 multimedia framework. Since we represent the depth video with a newly designed MPEG-4 Binary Format for Scene (BIFS) node and describe a dynamic 3D scene in the multimedia framework, the 3D video contents can be streamed to the client side through a network.