Constructing 3D natural scene from video sequences with vibrated motions

This paper presents a systematic approach to automatically construct 3D natural scenes from video sequences. Dense layered depth maps are derived from image sequences captured by a vibrated camera with only approximately known motion. The approach consists of (1) image stabilization by motion filtering and (2) depth estimation by spatio-temporal texture analysis. The two stage method not only generalized the so called panoramic image method and epipolar plane image method to handle image sequence vibrations due to the uncontrollable camera fluctuations, but also bypasses the feature extraction and matching problems encountered in stereo or visual motion. Our approach allows automatic modeling of the real environment for inclusion in VR representations.