Combining Spectral and Temporal Representations for Multipitch Estimation of Polyphonic Music

Due to the difficulty of creating pitch-labeled training data that cover the rich diversity found in music signals, unsupervised feature-based approaches derived from signal processing and feature design remain critical for multipitch estimation (MPE) of polyphonic music. While a large number of feature representations have been proposed in the literature, an effective means of combining different domains of features for MPE is still needed. In this paper, we propose a novel approach, referred to as combined frequency and periodicity (CFP), that detects pitches according to the agreement of a harmonic series in the frequency domain and a subharmonic series in the lag (quefrency) domain. This approach nicely aggregates the complementary advantages of the two feature domains in different frequency ranges, and improves the robustness of the pitch detection function to the interference of the overtones of simultaneous pitches. We report a comprehensive evaluation that compares CFP against three state-of-the-art approaches using three MPE datasets and four symphonies. The evaluation is characteristic of the coverage and complexity of music (in terms of instrument type and degree of polyphony). In addition, we also evaluate the performance of the MPE approaches when a number of audio degradations are applied. Results show that the proposed unsupervised method performs consistently well across the types of Western polyphonic music considered, and is robust to audio degradations such as high-pass filtering and MP3 compression.