Object Recognition Based on n-gram Expression of Human Actions

In this paper, we propose a novel method for recognizing objects by observing human actions based on bag-of-features. The key contribution of our method is that human actions are represented as n-grams of symbols and used to identify specific object categories. First, features of human actions taken on a object are extracted from video images and encoded to symbols. Then, n-grams are generated from the sequence of symbols and registered for corresponding object category. For recognition phase, actions taken on the object are converted into set of n-grams in the same way and compared with ones representing object categories. We performed experiments to recognize objects in an office environment and confirmed the effectiveness of our method.