Model-based margin estimation for hidden Markov model learning and generalisation

Recently, speech scientists have been motivated by the great, success of building margin-based classifiers, and have thus proposed novel methods to estimate continuous-density hidden Markov model (HMM) for automatic speech recognition (ASR) according to the notion that the decision boundaries determined by the estimated HMMs attain the maximum classification margin as in learning support vector machines. Although a good performance has been observed, the margin used in the ASR community is often specified as a parameter that has no explicit relationship with the HMM parameters. The issues of how the margin is related to the HMM parameters and how it directly characterises the generalisation capability of HMM-based classifiers have not been addressed so far in the community. In this study, the authors attempt to formulate the margin used in the soft margin estimation framework as a function of the HMM parameters. The key idea is to relate the standard distance-based margin with the concept of divergence among competing HMM state Gaussian mixture model densities. Experimental results show that the proposed model-based margin function is a good indication about the quality of HMMs on a given ASR task without the conventional needs of running experiments extensively using a separate set of test samples.