3D reconstruction of dynamic scenes from two asynchronous video-streams

We present an algorithm for reconstruction of dynamic scenes from a video input of an asynchronous stereo-pair. Our method presumes that the timestamps of the frame acquisitions and the intrinsic parameters of the stereo cameras are known, as well as, that the extrinsic parameters can be estimated from static regions of the scene. It computes the 3D trajectory of each individual point by modeling its motion parameters. Our algorithm requires in addition to the two asynchronous images from the camera pair, two additional images from one of the cameras for its processing. Since dynamic scene elements cause wrong reconstructions, here we introduce a two stage process in which the directions of the possible motion components are estimated prior to the 3D pose estimation of the corresponding point.