Pano-presence for teleoperation

Telepresence and teleoperation are immersive viewing and control by a user from a remote location. Usual implementations use a standard narrow field of view (FOV) camera and a communications link, and a head mounted display (HMD). Teleoperating a robotic vehicle or surveying a scene with such system and a computer monitor is difficult for human operators due to the narrow FOV of standard cameras, the unintuitive interface for directing the camera, and the loss of directional sense. For this reason these systems often use a head mounted display (HMD) instead of a monitor, however this introduces the HMD pose latency problem of latency and slow update due to the mechanical motion of the camera and the communications link. Even with good equipment, the experience is disorienting and slow. This paper proposes a pano-presence architecture for telepresence for applications such as teleoperation based on panoramic cameras, a communications link, and an HMD. The panoramic camera, capable of capturing light from all azimuth directions, provides a panorama which is be transported over the communications link to a panorama frame buffer for viewing in the HMD screen(s). The panorama viewing rate is decoupled from the communications latency so the user can look around freely without experiencing HMD pose latency problem, the delay in the HMD´s image alignment with the head position. A panorama frame format of an image cube is chosen since it can be viewed at full frame rate with the acceleration in consumer graphics cards. Two prototype systems, one telepresence and one teleoperation, using this architecture are described.