Applications of Recurrent Neural Network Language Model in Offline Handwriting Recognition and Word Spotting

The recurrent neural network language model (RNNLM) is a discriminative, non-Markovian model that can capture long-span word history in natural language. It has been proved to be successful in automatic speech recognition and machine translation. In this work, we applied RNNLM to the n-best rescoring stage of the state-of-the-art BBN Byblos OCR (optical character recognition) system for handwriting recognition.1 With RNNLM scores as additional features, our system achieved significant improvement (p <; 0.001), a 3.5% relative reduction on OCR word error rate, compared with a high baseline that uses n-gram language model for rescoring. We have also developed a novel method to integrate the OCR n-best RNNLM scores into the word posterior probabilities in OCR confusion networks, which resulted in consistent observable improvements in word spotting for OCR´ed handwritten documents, as measured by both mean average precision (MAP) and detection-error tradeoff (DET) curves.