Poster: Markerless fingertip-based 3D interaction for handheld augmented reality in a small workspace

Compared with traditional screen-touch input, natural gesture-based interaction approaches could offer a more intuitive user experience in handheld Augmented Reality (AR) applications. However, most gesture interaction techniques for handheld AR only use two degrees of freedom without the third depth dimension, while AR virtual objects are overlaid on a view of a three dimensional space. In this paper, we investigate a markerless fingertip-based 3D interaction method within a client-server framework in a small workspace. Our solution includes seven major components: (1) fingertip detection (2) fingertip depth acquisition (3) marker tracking (4) coordinate transformation (5) data communication (6) gesture interaction (7) graphic rendering. We describe the process of each step in details and present performance results of our prototype.