Ispy-usign humanoid assisted interactive sign language tutoring games

The work presented in this paper was carried out within an on-going project which aims to assist in Sign language tutoring to children with communication problems by means of interaction games with a humanoid robot. In this project, the robot is able to express words in the Turkish Sign language (TSL) and American Sign Language (ASL), and having comprehended the word, the child is motivated to give relevant feedback to the robot with colored flashcards, signs and sound. Within the multi-modal turn-taking games, child both learns the semantic and the realization of the words, besides have chance to test this information through imitation based interaction games. The performance of the children is also verified with flash card based activities as a part of the game, as a feedback for the teacher/parent. Child´s performance is recognized and evaluated by the robot through an RGB-D camera based system. This paper summarizes a longitudinal exploratory study from a classroom work, where children were taught 6 gestures including signs from ASL and TSL with the assistance of a Nao H25 humanoid robot. The performances of the children and robot are presented and discussed in terms of children´s subjective and objective evaluations.