A hybrid large vocabulary handwritten word recognition system using neural networks with hidden Markov models

We present a hybrid recognition system that integrates hidden Markov models (HMM) with neural networks (NN) in a probabilistic framework. The input data is processed first by a lexicon-driven word recognizer based on HMMs to generate a list of the candidate N-best-scoring word hypotheses as well as the segmentation of such word hypotheses into characters. An NN classifier is used to generate a score for each segmented character and in the end, the scores from the HMM and the NN classifiers are combined to optimize performance. Experimental results show that for an 80,000-word vocabulary, the hybrid HMM/NN system improves by about 10% the word recognition rate over the HMM system alone.