A real-time system for 3D recovery of dynamic scene with multiple RGBD imagers

In this paper we present a real-time 3D recovery system using multiple FPGA-based RGBD imagers. The RGBD imager developed in our lab produces color images combined with corresponding dense disparity maps encoding 3D information. Multiple RGBD imagers are externally triggered to sense the 3D world synchronously. The acquired 3D information of a dynamic scene from multiple viewpoints is then streamed to a PC cluster for further processing. Multiple RGBD imagers are fully calibrated to normalize the 3D data into a uniform world coordinate system. Compared with most of the visual hull based real-time 3D reconstruction systems, our system relies on depth maps and is much more suited for large scale dynamic scenes with multiple moving objects (e.g. people). Examples are provided to demonstrate the effectiveness of our system.