Score guided audio restoration via generalised coupled tensor factorisation

Generalised coupled tensor factorisation is a recently proposed algorithmic framework for simultaneously estimating tensor factorisation models where several observed tensors can share a set of latent factors. This paper proposes a model in this framework for coupled factorisation of piano spectrograms and piano roll representations to solve audio interpolation and restoration problem. The model incorporates temporal and harmonic information from an approximate musical score (not necessarily belonging to the played piece), and spectral information from isolated piano sounds. The performance of the proposed approach is evaluated on the restoration of classical music pieces where we get about 5dB SNR improvement when 50% of data frames are missing.