Integrating text and phonetic information for robust statistical speech translation

This paper focuses on the use of both text and phonetic information in a speech translation system in order to make translation results more robust to speech recognition errors. Conventional statistical speech translation formulas are extended to exploit both text-form and phonetic speech recognition results. A novel data-driven word/text tying algorithm is then proposed to group words based on both pronunciation similarity and meaning equivalency. In our speech-to-text translation experiments, significant improvement was achieved by using phonetic information and the proposed word tying algorithm.