A bounded trust region optimization for discriminative training of HMMS in speech recognition

In this paper, we have proposed a new method to construct an auxiliary function for the discriminative training of HMMs in speech recognition. The new auxiliary function serves as a first-order approximation of the original objective function but more importantly it remains as a lower bound of the original objective function as well. Furthermore, the trust region (TR) method in [1] is applied to find the globally optimal point of the new auxiliary function. Due to its lower-bound property, the found optimal point is theoretically guaranteed to increase the original discriminative objective function. The proposed bounded trust region method has been investigated on two LVCSR tasks, namely WSJ-5k and Switchboard 60-hour subset tasks. Experimental results show that the bounded TR method yields much better convergence behavior than both the conventional EBW method and the original TR method.