Bilingual Recurrent Neural Networks for improved statistical machine translation

Recurrent Neural Networks (RNN) have been successfully applied for improved speech recognition and statistical machine translation (SMT) for N-best list re-ranking. In SMT, we investigate using bilingual word-aligned sentences to train a bilingual recurrent neural network model. We employ a bag-of-word representation of a source sentence as additional input features in model training. Experimental results show that our proposed approach performs consistently better than recurrent neural network language model trained only on target-side text in terms of machine translation performance. We also investigate other input representation of a source sentence based on latent semantic analysis.