A real-time coarse-to-fine multiview capture system for all-in-focus rendering on a light-field display

We present an end-to-end system capable of real-time capturing and displaying with full horizontal parallax high-quality 3D video contents on a cluster-driven multiprojector light-field display. The capture component is an array of low-cost USB cameras connected to a single PC. Raw M-JPEG data coming from the software-synchronized cameras are multicast over Gigabit Ethernet to the back-end nodes of the rendering cluster, where they are decompressed and rendered. For all-in-focus rendering, view-dependent depth is estimated on the GPU using a customized multiview space-sweeping approach based on fast Census-based area matching implemented in CUDA. Realtime performance is demonstrated on a system with 18 VGA cameras and 72 SVGA rendering projectors.