Human-robot information sharing with structured language generation from probabilistic beliefs

This paper presents a framework for information sharing and fusion in cooperative tasks involving humans and robots. In this context, all information regarding the state of interest is recursively fused and maintained by each agent in a form of belief. For a robot agent, its belief is commonly and practically represented as a probability density function (pdf), formed by traditional sensor fusion and state estimation algorithms. In cooperative tasks with non-expert humans, a robot needs to effectively communicate its belief so that the gathered information can be easily processed and interpreted by the humans. The goal of this research is to provide two-way information exchange and fusion between robots and humans, the former operating on pdfs, while the latter on English sentences. This is achieved by considering two goodness measures: semantic correctness and information preservation. Based on the goodness measures studied, results show that the proposed framework is able to generate optimal statements describing the given belief pdfs and successfully recover the initial inputs used to generate them. Additionally, in order to describe complex belief pdfs, a Mixture of Statements (MoS) model is proposed such that the optimal expression can be generated through a composition of more than one statements. With a nonparametric Dirichlet Process MoS generation, it is found that the robot can determine correctly the number of statements as well as the corresponding reference parameters needed to describe all hypotheses underlying its belief.