A computer vision tangible user interface for mixed reality billiards

Conventional input devices such as the mouse and keyboard are un-natural and limited for various forms of human-computer interaction. We have created a tangible user interface that mixes the physical world with a virtual reality billiards game. Our system uses computer vision techniques to analyze images acquired from a single inexpensive digital video camera in real-time in order to passively sense the full 3D pose of a markerless cue stick. A physically-modeled billiards game updates the virtual cue stick to mimic detected poses and shots. Our system affords users with a convenient, noninvasive, inexpensive, and natural interface for entertainment or training without requiring a billiards table installation. Moreover, our system is robust to changes in illumination and occlusion, and is noteworthy for its use of computer vision as the sole means of sensing user input.