Visual interface for conducting virtual orchestra

A real-time visual recognition system, that enables a human conductor to control an electronic orchestra using gestures of a traditional conductor´s baton, is described. The positions of the baton and conductor´s hand are identified in a sequence of images from a pair of cameras, and tracked in 3D space. Gestures defining the musical beat are detected in the baton´s trajectory, and conveyed to a sound synthesis system, as events that control the tempo and the phase of the music. Parameters that can enable the control of the volume of sound are computed from the range variations of the baton and the hand. The system´s response is nearly instantaneous. The beat detection is reliable, and precise enough to be used by a professional conductor. It has been used to conduct an integrated electronic performance that combined a synthesized orchestra and animated ballet