A programming by demonstration system for human-robot collaborative assembly tasks

Programming by demonstration (PbD) has been one of the promising approaches to robot programming, where a robot learns how to operate by observing human demonstrations. While most of existing PbD research dealt with single-agent tasks, this paper deals with collaborative tasks, in which a robot and a human collaborate to assemble a structure by coordinating their operations. The system has three parts: demonstration observation parts, task modeling parts, and task execution parts. A scene recognition system is developed for reliable action recognition in human demonstrations. The developed PbD system has successfully been applied to a collaborative assembly tasks with more than twenty assembly steps.