Interactive machine learning for incorporating user emotions in automatic music harmonization

Harmonization enriches piano melodies by adding variations such as mood, sound enhancements and beats that are the key building blocks of piano music. However, not all piano players and song writers are gifted with the musical talent of harmonizing piano melodies effectively since it requires keeping track of an extensive set of western music rules and concepts, years of training and practice and also musicality within them to harmonize a melody accurately. This paper discusses a solution for the tedious task of harmonization by introducing `ChordATune´, an interactive tool for harmonizing melodies and generating chord progressions according to user emotions. Further, ChordATune provides a mechanism to arrange chords according to different genres, drum beats and tempi based on user preference. A machine learning approach with Hidden Markov Model (HMM), along with dynamic programming is used to generate the chord progression for a given melody and embed the emotional factor of the user. The melody is taken in as an audio file to the system, where a pitch class profile is created at run time representing the pitch content of the file over time. In order to embed the emotional factor, the Hidden Markov Model is dynamically created, and HMM properties are generated at run time according to the selected emotional factor and the input pitch classes (melody). Around 250 lead sheets were used to train the system using data driven and heuristic approaches, and the evaluation results represented 80% user satisfaction of the prototype. This research further opens a path for research concerning chord progression generation for vocals, taking into account the extraction of words, emotional factor and the tune extracted from the actual voice of the user.