Speech recognition performance assessments and available databases

Measuring the performance of speech recognition systems is integral to advancing the technology as well as applying it. Appropriately designed controlled experiments are the necessary tools by which both research and commercial systems must assess the value of different algorithms and approaches. Furthermore, they form the foundation upon which potential consumers of this technology may base sound price- performance decisions for commercial applications. The objectives of this presentation are: 1) to review the major performance assessment activities presently underway, especially those of the National Bureau of Standards (U.S. Dept. of Commerce) and the NATO Research Study Group on Speech Recognition, 2) to present a set of recommendations and evaluation guidelines recently compiled from speech researchers based in academia, industry, and government, 3) to reference a number of specific evaluation proposals generated within the speech community, and 4) to summarize and briefly describe a compilation of speech data bases which under certain conditions have been or may become available.