Visualization and simulation of sensory events as a representation of states for state-based teaching by demonstration in VR

"Teaching by demonstration" is a method to generate a robot program that makes a robot do the same task as the task that a human operator demonstrates. We have developed a "teaching by demonstration in VR" system (TbDinVR) which automatically generates a robot program to work in the real world after a task is demonstrated by an operator in the virtual world. In this paper, we have explained necessary and sufficient condition for states in a task description, advantages of using state-based TbDinVR system, and a simulator for determining parameters of skill primitives by using sensor simulation in VR and a random search technique in detail. We have shown an experimental result and visualization of that result.