3D tracking of multi-objects using color and stereo for HCI

We present a 3D tracking method of multi-objects by color and stereo. The results are applied to navigation and manipulation in a virtual environment. We choose a human face and hands as tracking targets for HCI. To extract the area of face and hands in a complex background, we transform an input color image using the GSCD (generalized skin color distribution). Based on the transformed image, we detect the area of face and hands by WUPC. Furthermore, the face and hands candidate region are estimated by a Kalman filter in the following frame. Then we can extract the depth information about the only corresponding region, respectively, by stereo matching. Finally, we can navigate and manipulate objects based on the extracted 3D information of face and hands naturally in the virtual environment without any additional device