Hierarchical hidden Markov modeling for team-play in multiple agents

When we consider imitation learning of team-play in multi-agent systems, we need to define a suitable building block and its interface to construct complex joint behaviors. I focus on agent intentions as the building block that abstracts local situations of the agent, and propose a hierarchical hidden Markov model (HMM). The key of the proposed model is introduction of gate probabilities that restrict transitions among agents´ intentions according to others´ intentions. Using these probabilities, the framework can control transitions flexibly among basic behaviors in a cooperative behavior. Experiments shows that the model can acquire suitable timing to change an agent´s intention cooperatively with other agents.