A Framework for Tangible User Interfaces within Projector-based Mixed Reality

This paper proposes a framework named TIPMR, for designing tangible user interfaces (TUIs) within projector-based mixed reality applications. The framework divides the target application into three parts: GUI-based application, TUI and an assistant. The assistant is employed as an adapter to translate between TUI operations and general GUI commands like mouse or keyboard events. This architecture makes it easier to focus on designing GUI-based applications and TUI separately. We built a tourist guidance system with two different tangible interaction modes based on TIPMR to demonstrate its usefulness and efficiency.