Unsupervised learning of multi-hypothesized pick-and-place task templates via crowdsourcing

In order for robots to be useful in real world learning scenarios, non-expert human teachers must be able to interact with and teach robots in an intuitive manner. One essential robot capability is wide-area (mobile or nonstationary) pick-and-place tasks. Even in its simplest form, pick-and-place is a hard problem due to uncertainty arising from noisy input demonstrations and non-deterministic real world environments. This work introduces a novel method for goal-based learning from demonstration where we learn over a large corpus of human demonstrated ground truths of placement locations in an unsupervised manner via Gaussian Mixture Models. The goal is to provide a multi-hypothesis solution for a given task description which can later be utilized in the execution of the task itself. In addition to learning the actual arrangements of the items in question, we also autonomously extract which frames of reference are important in each demonstration. We further verify these findings in a subsequent evaluation and execution via a mobile manipulator.