Automatic performance assessment of speech recognisers through telephone quality speech databases

Performance assessment of speech recognisers is one of the major area of interest in the field of speech processing technology. The reasons for such an interest stem from the intrinsic difficulty of the task in terms of repeatability, reliability and validity of the results. It is in fact meaningless to declare any performance measurement without specifying the boundary conditions in which those results were achieved. This consideration becomes even more crucial when the telephone environment is concerned. The authors cover two separate aspects: the generation of telephone grade speech corpora and the assessment of a speech recogniser with both clean and telephone grade speech corpus, the latter being derived from the clean speech version. For the first aspect, the authors describe a complete procedure to generate telephone grade speech corpora; it could be used either to generate completely new corpora or to degrade a given clean corpus to its telephone grade equivalent. Concerning the second aspect, the authors focus on a European standard multilingual corpus, the EUROM 0 CD ROM that has been used twice for a comparative assessment with the same speech recogniser