Model-based non-negative matrix factorization for single-channel speech separation

A model-based non-negative matrix factorization (NMF) algorithm is formulated for single-channel speech source separation. With linguistic priors of the speech sources, the state-aligned spectral envelopes of these sources are inferred from acoustic models. Being initialized with the computed spectral envelopes, NMF processing is implemented to estimate the spectral envelope trajectory of each source. Subsequently the source speech is generated by reshaping the mixture spectra according to the estimated spectral envelopes, followed by the reduction of interfering harmonic components. An iterative process is developed for more reliable time alignment and hence better performance of separation. Experimental results show that two speech sources with equal intensity can be successfully separated by the proposed algorithm.