Margin-Based Discriminative Training for String Recognition

Typical training criteria for string recognition like for example minimum phone error (MPE) and maximum mutual information (MMI) in speech recognition are based on a (regularized) loss function. In contrast, large-margin classifiers-the de-facto standard in machine learning-maximize the separation margin. An additional loss term penalizes misclassified samples. This paper shows how typical training criteria like for example MPE or MMI can be extended to incorporate the margin concept, and that such modified training criteria are smooth approximations to support vector machines with the respective loss function. The proposed approach takes advantage of the generalization bounds of large-margin classifiers while keeping the efficient framework for conventional discriminative training. This allows us to directly evaluate the utility of the margin term for string recognition. Experimental results are presented using the proposed modified training criteria for different tasks from speech recognition (including large-vocabulary continuous speech recognition tasks trained on up to 1500-h audio data), part-of-speech tagging, and handwriting recognition.