On the Relationship Between Bayes Risk and Word Error Rate in ASR

Recently, a number of (approximate) approaches emerged in speech processing, which try to overcome the known lack of match between symbol level evaluation measures (e.g., word error rate) and the standard string (symbol sequence) cost (e.g., sentence error)-based Bayes decision rule, by using symbol level cost functions for Bayes decision rule. Nevertheless, experiments show that for a majority of test samples both decision rules still give equal decisions, especially at lower error rates. In this paper, analytic evidence for these observations is provided. A set of conditions is presented, for which Bayes decision rule with symbol level and string level cost function leads to the same decisions. Furthermore, the case of word error cost represented by the Levenshtein (edit) distance is investigated, which upon others covers the important case of speech recognition. A Hamming distance-based upper bound to the Levenshtein cost function is discussed. This cost function relates to former, word-posterior based decision rules, and the corresponding efficient decision rule is shown to be strongly related to Bayes decision rule with the Levenshtein cost. The analytic results are verified experimentally, and their quantitative effect is studied by experiments on four different well-known large vocabulary automatic speech recognition tasks.