A combined studio production system for 3-D capturing of live action and immersive actor feedback

A new studio system for the production of three-dimensional (3-D) content is introduced. The system combines the ability to capture dynamic scenes, based on a multicamera system in a chroma-key environment, with a view-dependent projection for actor feedback. The system allows the generation and rendering of 3-D models in preview quality for on-set visualization in real time and in high quality for postproduction applications in an offline phase. The shape reconstruction is based on the computation of the visual hull. The view-dependent projection component allows actors to be immersed into a virtual scene and to interact with virtual components. The functional modules of the system were implemented in a highly distributed system using standard, inexpensive IT components. Therefore, the system is quite scalable and can be fitted into most studio environments. For the integration of the system components, a middleware architecture was developed.