Computer Vision Method in Music Interaction

This paper presents a method based on computer vision, which allows the gesture recognition of a pianist´s right hand. The choice of computer vision as a processing method was made due to the lack of a satisfying fingering detection system addressing musicians. On the other hand, hidden Markov models (HMM) are an advisable mathematic theory managing stochastic processes such as human gestures. Both HMM and computer vision co-operate to achieve content-based video retrieval in music interaction, as presented in this paper. Thus, we developed a method of hand extraction, based on the special position assumed by the hand of the pianist. The study of the pianist fingering retrieval has been effectuated on four aspects: preparation and capture of video signal, hand segmentation, finger extraction, feature vector exportation and HMM classifier.