Vision-based Human Modeling and recognition for human system interaction

In order to ensure smooth interactions with humans, a system needs to provide appropriate services based on a clear understanding of the capabilities, characteristics, and conditions of humans. We aim to acquire qualitative and quantitative information on humans by using image-sensing technology for human observation to realize even more advanced human-system interactions. For sensing of human users, it is critical that information be acquired robustly from features available in images and from a prior knowledge model of the human body. We are conducting various research studies with a view to establishing human sensing technology capable of extracting and expressing human features while responding flexibly to the individual differences and fuzziness inherent in humans. In this paper, we describe modeling of human shape and behavior based on combined use of image information and a human body model (Vision-based Human Modeling) and human behavior recognition in video (Vision-based Human Recognition). We demonstrate the application of these results to human-system interaction.