Video stabilization for HMD-based telexistence - Concept and prototype configuration

This paper proposes a method to stabilize video images for head-mounted display (HMD)-based telexistence, to alleviate the simulator sickness experienced by the user when considerable end-to-end delay exists. Stabilization can be achieved by shifting the displayed image such that the direction from the user´s eye to the object image in the visual display is aligned with that from the camera to the real object in the remote environment. The amount of shift is calculated by comparing the orientation of the head of the robot at the moment when the camera of the robot acquired the video frame and that of the user´s head when the corresponding frame is being displayed. As the first step of our study, we built a system consisting of a motorized camera head and an HMD in conjunction with software to display the shifted image in the HMD. We verified that master-slave motion and the display of the real-time shifted image on the HMD were satisfactory.