Interactive Sound Generation System Based on Instruction Recognition of Human Body Motion

Recently, the interactive sound is researched in the field of an artificial life or Kansei engineering as one of interactive arts. An interactive sound changes the aspect variously by the interaction of human and a system, and it aims at generating the sound which was full of the complexity and diversity beyond expectations of human. In our laboratory, the chaotic sound generation system which shows various aspects has been developed by using the Globally Coupled Map (GCM) which puts plural chaos elements in order and operates each chaos and the whole synchronicity. However, in the present system, what we can operate is only a chaos parameter. Therefore this is the system which it is hard to reflect the human´s intention in the generated sound. So, in this research, the system which analyzes human motions and instructions with an external camera and defines the directivity of the output sound is proposed. Motion capture system detects coordinates of human´s joint. We designed the sound generation system to which the element of sound pitch, length, volume, and tonality are changed by the detected motion. The intention of desired sound by human is able to be reflected to the system, and we can expect to realize the large interaction effect. Moreover, it is also reported by Kansei evaluation to confirm the efficiency of the designed system.