Agents that talk and hit back: animated agents in augmented reality

AR puppet is a hierarchical animation framework for augmented reality agents, which is a research area combining augmented reality (AR), sentient computing and autonomous animated agents into a single coherent human-computer interface paradigm. While sentient computing systems use the physical environment as an input channel, AR outputs virtual information superimposed on real world objects. To enhance man-machine communication with more natural and efficient information presentation, this framework adds animated agents to AR applications that make autonomous decisions based on their perception of the real environment. These agents are able to turn physical objects into interactive, responsive entities collaborating with both anthropomorphic and non-anthropomorphic virtual characters, extending AR with a previously unexplored output modality. AR puppet explores the requirements for context-aware animated agents concerning visualization, appearance, behavior, in addition to associated technologies and application areas. A demo application with a virtual repairman collaborating with an augmented LEGO® robot illustrates our concepts.