Towards interactive physical robotic assistance: Parameterizing motion primitives through natural language

Natural language interaction between humans and robots is a very challenging topic, especially when it refers to motion descriptions in a certain environment. This problem is particularly relevant during physical human-robot interaction, e.g. in cooperative transportation tasks, where the partners´ physical coupling requires an agreement on the way to follow. Understanding in depth the link between sentences, words, environmental properties and motions can deeply enhance the interaction between humans and robots. In this work, we propose a novel approach for learning relations and dependencies between motion, natural language and environmental properties using parameterized left-to-right time-based Hidden Markov Models. A natural language model represents the link between language and motion symbols while the HMMs parameterization corresponds to the explicit influence on motions of both words and environmental features. The proposed PHMM approach parameterizes the output and the transition probabilities using a non-linear dependency estimation. The method is validated by learning and generating navigation primitives in a 2 Degrees-Of-Freedom (DoF) virtual scenario.