Integrating a Real-Time Captured Object into Mixed Reality

Web-based mixed reality (MR) that combines remote real objects with a virtual environment to form a realistic 3D MR space has received much interest in cyberworlds. This paper introduces a novel method of integrating a live video object that is taken by multiple fixed cameras in a remote natural environment into a MR space. Using the captured live video frames, the view of the remote object corresponding to the user´s current viewpoint in the MR space is generated in realtime based on the proposed fast view morphing algorithm, which ensures a natural image transition without reconstructing the 3D model of the object. During the user´s free navigation in the MR space, the real-time generated view of the object is presented in that space, which results in a convincing impression that the remote object exists in the local realistic 3D MR space. Experimental results in both a desktop MR system and a CAVE-like MR system have demonstrated the validity of the proposed approach