Voice conversion based on style and content separation with dual latent variable model

This paper presents a novel method for voice conversion based on style and content separation, which is solved by using dual latent variable model (D-LVM). Based on D-LVM, the vocal tract spectrum of speech represented by line spectral frequencies (LSF) is explicitly decomposed into so-called style and content factors, which are used to represent the speech meaning and the speaker individuality respectively. On the basis of reasonable separation of style and content for speech, voice conversion is performed successfully by reproducing converted speech using the initial speech content and the target speaker style. The objective and subjective tests show that, under the condition of limited training dataset, the method proposed in the paper gets better conversion performance compared to the conventional mapping based GMM system and SVD based bilinear model.