Thermal touch: Thermography-enabled everywhere touch interfaces for mobile augmented reality applications

We present an approach that makes any real object a true touch interface for mobile Augmented Reality applications. Using infrared thermography, we detect residual heat resulting from a warm fingertip touching the colder surface of an object. This approach can clearly distinguish if a surface has actually been touched, or if a finger only approached it without any physical contact, and hence significantly less heat transfer. Once a touch has been detected in the thermal image, we determine the corresponding 3D position on the touched object based on visual object tracking using a visible light camera. Finally the 3D position of the touch is used by human machine interfaces for Augmented Reality providing natural means to interact with real and virtual objects. The emergence of wearable computers and head-mounted displays desires for alternatives to a touch screen, which is the primary user interface in handheld Augmented Reality applications. Voice control and touchpads provide a useful alternative to interact with wearables for certain tasks, but particularly common interaction tasks in Augmented Reality require to accurately select or define 3D points on real surfaces. We propose to enable this kind of interaction by simply touching the respective surface with a fingertip. Based on tests with a variety of different materials and different users, we show that our method enables intuitive interaction for mobile Augmented Reality with most common objects.