Title :
Confidence measures for dialogue management in the CU Communicator system
Author :
San-Segundo, Ruben ; Pellom, Bryan ; Ward, Wayne ; Pardo, José M.
Author_Institution :
Center for Spoken Language Understanding, Colorado Univ., Boulder, CO, USA
Abstract :
This paper provides improved confidence assessment for detection of word-level speech recognition errors and out-of-domain user requests using language model features. We consider a combined measure of confidence that utilizes the language model back-off sequence, language model score, and phonetic length of recognized words as indicators of speech recognition confidence. The paper investigates the ability of each feature to detect speech recognition errors and out-of-domain utterances as well as two methods for combining the features contextually: a multi-layer perceptron and a statistical decision tree. We illustrate the effectiveness of the algorithm by considering utterances from the ATIS airline information task as either in-domain and out-of-domain for the DARPA Communicator task. Using this hand-labeled data, it is shown that 27.9% of incorrectly recognized words and 36.4% of out-of-domain phrases are detected at a 2.5% false alarm rate
Keywords :
decision trees; interactive systems; speech recognition; speech-based user interfaces; ATIS airline information task; CU Communicator system; DARPA Communicator task; back-off sequence; confidence measures; dialogue management; incorrectly recognized words; language model features; language model score; multi-layer perceptron; out-of-domain phrases; out-of-domain user requests; out-of-domain utterances; phonetic length; recognized words; speech recognition confidence; statistical decision tree; word-level speech recognition errors; Computer vision; Context; Decision trees; Decoding; Length measurement; Multilayer perceptrons; Natural languages; Power system modeling; Robustness; Speech recognition;
Conference_Titel :
Acoustics, Speech, and Signal Processing, 2000. ICASSP '00. Proceedings. 2000 IEEE International Conference on
Conference_Location :
Istanbul
Print_ISBN :
0-7803-6293-4
DOI :
10.1109/ICASSP.2000.859190