Automated assembly skill acquisition through human demonstration

Acquiring robot assembly skills through human demonstration is a challenging problem. To achieve this goal, not only the actions and objects have to be shown to the robot, but also the effect of the action needs to be estimated. Recognizing the subtle assembly actions is a non-trivial task, and it is difficult to estimate the effect of the action on the assembly parts due to the small part sizes. In this paper, with a RGB-D camera, we build a Portable Assembly Demonstration (PAD) system which can recognize the part/tool used, the action applied and the assembly state characterizing the spatial relationship between the parts. The experiment results proved that this PAD system can generate an assembly script with good accuracy in object and action recognition as well as assembly state estimation.