A unified representation for reasoning about robot actions, processes, and their effects on objects

Mobile manipulation robots are becoming more and more common and begin to extend their task spectrum towards more general housework activities. The sequence of actions needed to accomplish such tasks can be obtained from instructions on the Internet originally written for humans. While giving valuable information about the types of actions and some of their parameters, these instructions usually lack information that humans consider to be obvious. In this paper, we investigate how we can equip robots with sufficient knowledge and inference mechanisms to competently detect and fill such knowledge gaps in descriptions of everyday activities. We present methods for projecting the effects of actions and processes, for inferring action parameters like the objects and locations to be used, and introduce representations for reasoning about object transformations resulting from the effects of actions.