A fast and robust fingertips tracking algorithm for vision-based multi-touch interaction

Finger touch is the most natural way for human interaction with the external world. In the past five years, the great success of multi-touch trackpad on portable devices implies the big potential of multi-touch technology to be applied in vision-based human-computer interaction (HCI) systems. The implementation of multi-touch technology highly depends on accurate and fast fingertip tracking. In this paper we present a fast and robust algorithm for tracking fingertip positions in a stereovision-based 3D multi-touch interaction system. Our method first detects the hand region by a two-step strategy based on skin color filter as well as depth images. Then a geometry model is built to locate the fingertips. The accuracy and effectiveness of the fingertip tracking algorithm is examined over several video sequences with complicated backgrounds. Experimental results verify that our algorithm can reliably and accurately track the movements of fingertips in real time. The effectiveness of the fingertip tracking algorithm also reveals the capability allowing user to interact with computers through their finger movements in 3D space over a virtual thin-film-like touch surface which is set up at a certain distance away from the screen.