Freischutz digital: A multimodal scenario for informed music processing

In the last decade there has been an explosion in the availability of digitized music material, which comprises data of various formats and modalities including textual, symbolic, acoustic and visual rep-resentations. For example, in the case of an opera there typically exist digitized versions of the libretto, different editions of the musical score, as well as a large number of performances given as audio and video recordings. In this paper, we give an overview of various informed approaches to music processing, where the availability of multiple sources of music-related information is used for supporting and improving the analysis of music data. Considering the scenario of the opera “Der Freischüitz” by Carl Maria von Webera work of central musical importance, where one can draw upon a rich body of sources-we highlight how the identification and creation of cross-modal relationships are a key issue in multimedia processing.