A Singing Voice Synthesizer Controlled by the Arm Motions

A singing voice synthesizer controlled by user´s arm motion is proposed for an entertainment tool. The system measured three dimensional position of the hand and wrist angle using magnetic motion sensor. The measured values determine loudness, pitch, depth of vibrato, and phoneme for the synthesized vowel-like sound. In order to make users synthesize high-quality sound without long training, the system employs an intuitive mapping from the motion to sound parameters. Two evaluation experiments were carried out. The first experiment revealed that users could learn to control loudness and pitch using this mapping in 23 minutes on the average. The second experiment indicated a quantization process in pitch determination to be effective to improve perceptual quality of the synthesized sound.