Improved depth propagation for 2D to 3D video conversion using key-frames

The success of introducing 3D television will depend on the availability of high-quality 3D video. A possible solution to this problem is to convert existing 2D content into 3D. We are working on methods to do this with minimal user interaction and therefore minimal cost. In this paper we improve our previous depth propagation that is based on bilateral filtering. The proposed improved method uses a two stage approach, where first a per-pixel depth estimate is done using a bilateral filter after which in a second step this estimate is corrected through a block-based motion compensation from the previous frame. The best fitting block of the previous frame thus overwrites the depth estimate based on the bilateral filtering. The proposed method raises the average Peak- Signal-To-Noise ratio of newly rendered off-centre view images. The improvement over 99 propagated frames ranges from 0.2 up to 4.3 dB depending on the image sequence.