What do you expect from a robot that tells your future? The crystal ball

This paper proposes an approach to hierarchy formation of human behaviors, extraction of the behavioral transitions, and their application to prediction and automatic generation of behaviors. Human demonstrator motion patterns are stored as motion symbols, which abstract the motion data by using Hidden Markov Models. The stored motion patterns are organized into a hierarchical tree structure, which represents the similarity among the motion patterns and provides abstracted motion patterns. Concatenated sequences of motion patterns are stochastically represented as transitions between the abstracted motion patterns by using an Ngram Model, and the transitional relationships of the human behaviors are extracted. The behavioral hierarchy and transition model make it possible to predict human behaviors during observation and to generate sequences of motion patterns automatically while maintaining a natural motion stream, as if the system is a “crystal ball” to reflect future behaviors. The experiments validates the proposed framework by using a developed visualization system, which shows the demonstrator or the operator the established hierarchical tree and the transition network of the motion patterns, predicted behaviors and generated sequences of the motion patterns.