Gesture recognition using combination of acceleration sensor and images for casual communication between robots and humans

A fuzzy logic based multi-modal gesture recognition method is proposed, where both camera images and hand motion data (given by a 3D acceleration sensor put on human wrists) are used to notify the emotion of humans to robots in real time. To demonstrate the validity, it is applied to a part of the home party scenario enjoyed by five eye robots and four human participants, where 8 types of human emotional-gestures are successfully recognized by robots. The proposed method aims to realize the casual communication between robots and humans in the mascot robot system that has been developed by authors´ group.