A Quadratic Optimization Approach to Discriminative Training of CDHMMs

In this letter, we reformulate the discriminative training (DT) of continuous density hidden Markov models (CDHMMs) as an ellipsoid constrained quadratic programming (ECQP) problem, and we solve it by a line search algorithm. The ellipsoid constraint intrinsically arises from the step-size control in each iteration of DT optimization, which leads to an efficient solution without relaxing the objective function to be convex, as in a general quadratic programming problem. Moreover, the problem can be equivalently converted to a lower-dimensional one under some conditions, which helps further to simplify the solution. We show that under a Kullback-Leibler divergence(KLD) constraint, DT of CDHMM parameters such as Gaussian means and variances can be efficiently solved by the proposed algorithm, with only mild assumptions adopted. Experimental results on two tasks show that the ECQP approach considerably outperforms other popular algorithms in terms of both final recognition accuracy and convergence speed.