English to Japanese spoken lecture translation system by using DNN-HMM and phrase-based SMT

This paper presents our scheme to translate spoken English lectures into Japanese that consists of an English automatic speech recognition system (ASR) that utilizes a deep neural network (DNN) and an English to Japanese phrase-based statistical machine translation system (SMT). We utilized an existing Wall Street Journal corpus for our acoustic model and adapted it with MIT OpenCourseWare lectures whose transcriptions we also utilized to create our language model. For the parallel corpus of our SMT system, we used TED Talks and Japanese News Article Alignment Data. Our ASR system achieved a word error rate (WER) of 21.0%, and our SMT system achieved a 3-gram base bilingual evaluation understudy (BLEU) of 16.8 for text input and 14.6 for speech input, respectively. These scores outperformed our previous system : WER = 32.1% and BLEU = 11.0.