Learning to refine behavior using prosodic feedback

We demonstrate the utility of speech prosody as a feedback mechanism in a machine learning system. We have constructed a reinforcement learning system for our humanoid robot Nico, which uses prosodic feedback to refine the parameters of a social waving behavior. We define a waving behavior to be an oscillation of Nico´s elbow joint, parameterized by amplitude and frequency. Our system explores a space of amplitude and frequency values, using q-learning to learn the wave which optimally satisfies a human tutor. To estimate tutor feedback in real-time, we first segment speech from ambient noise using a maximum-likelihood voice-activation detector. We then use a k-Nearest Neighbors classifier, with A=3, over 15 prosodic features, to estimate a binary approval/disapproval feedback signal from segmented utterances. Both our voice-activation detector and prosody classifier are trained on the speech of the individual tutor. We show that our system learns the tutor´s desired wave, over the course of a sequence of trial-feedback cycles. We demonstrate our learning results for a single speaker on a space of nine distinct waving behaviors.