Title :
Variational Bhattacharyya divergence for hidden Markov models
Author :
Hershey, John R. ; Olsen, Peder A.
Author_Institution :
T. J. Watson Res. Center, IBM, Yorktown Heights, NY
fDate :
March 31 2008-April 4 2008
Abstract :
Many applications require the use of divergence measures between probability distributions. Several of these, such as the Kullback-Leibler (KL) divergence and the Bhattacharyya divergence, are tractable for simple distributions such as Gaussians, but are intractable for more complex distributions such as hidden Markov models (HMMs) used in speech recognizers. For tasks related to classification error, the Bhattacharyya divergence is of special importance, due to its relationship with the Bayes error. Here we derive novel variational approximations to the Bhattacharyya divergence for HMMs. Remarkably the variational Bhattacharyya divergence can be computed in a simple closed-form expression for a given sequence length. One of the approximations can even be integrated over all possible sequence lengths in a closed-form expression. We apply the variational Bhattacharyya divergence for HMMs to word confusability, the problem of estimating the probability of mistaking one spoken word for another.
Keywords :
Bayes methods; hidden Markov models; speech recognition; Bayes error; Kullback-Leibler divergence; hidden Markov models; probability distributions; speech recognizers; variational Bhattacharyya divergence; Closed-form solution; Density measurement; Gaussian distribution; Gaussian processes; Hidden Markov models; Machine learning; Probability density function; Probability distribution; Speech recognition; Statistical distributions; Bhattacharyya Error; Bhattacharyya divergence; Gaussian mixture models (GMMs); hidden Markov models (HMMs); variational methods;
Conference_Titel :
Acoustics, Speech and Signal Processing, 2008. ICASSP 2008. IEEE International Conference on
Conference_Location :
Las Vegas, NV
Print_ISBN :
978-1-4244-1483-3
Electronic_ISBN :
1520-6149
DOI :
10.1109/ICASSP.2008.4518670