Interfacing sound synthesis to movement for exploring high-dimensional systems in a virtual environment

An auditory feedback plays an important role in our cognitive process. We want to make use of this sensory feedback when exploring high-dimensional systems. We present a manifold interface as a generalizable tool featuring three human performance abilities, to see, to move, and to listen. We visualize control parameter space the domain of which corresponds to the domain of input gestures. The gesture input is extended to a 3D visual representation of the control space in which one can efficiently apply multiple parameter variation techniques by performing movements. The consequences of parameter variation are computed in real-time sound synthesis with respect to the systems under study so that one can listen to the changes in the systems as they vary their motions. The paper introduces the working vocabulary control path, window space, out-of-time displacements, and in-time displacements. Applications of the interface are briefly discussed