Teaching a tele-robot using natural language commands

For Internet-based teleoperation systems, user-friendly natural interfaces are advantageous because those systems are intended to be used by non-experts. In developing user friendly interfaces, natural language communication is mandatory. This paper presents a system in which a sub-set of natural language is used to command a tele-robot manipulator doing an object sorting task. The paper discusses about referring to objects with natural language commands such as "pick the small red cube". This is achieved by learning individual lexical symbols that refer to colors, shapes, and sizes independently, and then inferring the meaning of a combination of them.