Towards On-Line Digital Doubles

We present a modular system for real-time 3D-scanning of human bodies under motion. The high-resolution shape and colour appearance is captured by several scanning units positioned around the object of interest. Each of these units performs a foreground-background segmentation and computes a valid depth-range for the spatially neighbouring units. Multiple depth-ranges are combined in a visual hull representation, which limits the search-range for the 3D-reconstruction. Depth-estimation is based on a hierarchical multi-view-stereo plane sweep approach. Robustness and accuracy is increased by incorporating imperceptible infrared illumination as well as adding local pixel gradient information. All parts of the processing pipeline, involving camera color conversions, segmentation, depth-range computation, visual-hull generation, lossless image compression, network transfer of the infrared and colour images, and the plane sweep algorithm, are implemented on the GPU and highly optimized for speed, allowing scanning times of less than 40 ms per frame. Experimental results demonstrate the applicability of our system to the creation of high-density on-line digital doubles.