High-Resolution Performance Capture by Zoom-in Pan-Tilt Cameras

We have developed a system with multiple pan-tilt cameras for capturing high-resolution videos of a moving person. This system controls the cameras so that each camera captures the best view of the person (i.e. one of body parts such as the head, torso, and limbs) based on criteria for camera-work optimization. For achieving this optimization in real time, time-consuming pre-processes, which give useful clues for the optimization, are performed in a training stage. Specifically, a target performance (e.g. a dance) is captured to acquire the configuration of the body parts at each frame. In a real capture stage, the system compares an online-reconstructed shape with those in the training data for fast retrieval of the configuration of the body parts. The retrieved configuration is used by an efficient scheme for optimizing a camera work. Experimental results show the camera work optimized in accordance with given criteria. A high-resolution 3D videos produced by the proposed system are also shown as a typical use of high-resolution videos.