Support Vector Machine Re-scoring of Hidden Markov Models

We present a method that uses a set of maximum-likelihood (ML) trained discrete HMM models as a baseline system, and an SVM training scheme to re-score the results of the baseline HMMs. It turns out that the re-scoring model can be represented as an un-normalized HMM. We refer to these models as pseudo-HMMs. The pseudo-HMMs are in fact a generalization of standard HMMs, and by proper discriminative training they can result in performance improvement compared to standard HMMs. We consider two SVM training algorithms. The first corresponds to the one against all method. The second corresponds to the one class transformation training method. The one class training algorithm can be extended to an iterative algorithm, similar to segmental K-means. In this case the final output of the algorithm is a single set of pseudo-HMMs. Although they are not normalized, this set of pseudo-HMMs can be used in the standard recognition procedure (the Viterbi recognizer), as if they were plain HMMs. We use an isolated noisy digit recognition task to demonstrate that SVM re-scoring of HMMs typically reduces the error rate significantly compared to standard ML training.