Document recognition and translation system for unconstrained Arabic documents

We describe an end-to-end system for translating real-world Arabic field documents that contain a mix of handwritten and printed content into English. These documents are extremely challenging to recognize due to presence of noise, poor image capture quality, and variations in writing style, writing device, font, layout, genre, etc. Furthermore, no off-the-shelf machine translation (MT) engine is available to translate these documents into English. We present key innovations for dealing with these challenges for document preprocessing, text line segmentation, and text recognition. In addition, we describe our approach for adapting MT using a limited amount of in-domain training data that results in significant improvements in translating accuracy.