Inferring narrative and intention from playground games

We present a system which observes humans participating in various playground games and infers their goals and intentions through detecting and analyzing their spatiotemporal activity in relation to one another, and then builds a coherent narrative out of the succession of these intentional states. We show that these narratives capture a great deal of essential information about the observed social roles, types of activity and game rules by demonstrating the systempsilas ability to correctly recognize and group together different runs of the same game, while differentiating them from other games. Furthermore, the system can use the narratives it constructs to learn and theorize about novel observations, allowing it to guess at the rules governing the games it watches. For example, after watching several different games, the system figures out on its own that Tag-like games require close physical proximity in order for the role of ldquoitrdquo to swap from one person to another. Thus a rich and layered trove of social, intentional and cultural information can be drawn out of extremely impoverished and low-context trajectory data.