3D video player system with haptic interaction based on depth image-based representation

In this paper, we propose a three-dimensional (3D) video player system that provides haptic interaction with objects in a video scene based on a depth image-based representation. In order to represent dynamic 3D scenes, 3D video media combining general color video and synchronized depth video (depth image sequences) containing per-pixel depth information were exploited. With the proposed system, viewers can touch and explore the shape of objects visible in a scene via a three degree-of-freedom (DOF) haptic interface. To support haptic interaction in the 3D video player, a modified proxy graph algorithm was proposed which calculates the contact force between the 3D video scene and the viewers´ finger. In addition, the proposed system architecture was implemented which encompassed 3D video contents generation, depth data enhancement, compression, and haptic rendering. The proposed 3D video player system showed stable, smooth, and immersive interaction in real-time.