Evaluation and error recovery methods of an IVR based real time speech recognition application

Field trial and evaluation of any real world speech recognition application using Interactive Voice Response technology are likely to be a daunting task. It has to face challenges regarding spoken language conventions, pronunciation variations, recognition issues in noisy environment, limitations of human cognition, working memory and differences between users. Present study illustrates the entire evaluation process of such an agricultural information retrieval system mainly targeted towards semi-literate or illiterate farmers. A new set of evaluation metrics as per the designed evaluation strategies, details of field trial processes, feedback analysis and finally system performance results are presented in a well organized way. Additionally to meet users´ expectations, distinctive error recovery methods like Signal Analysis and Decision, Confidence Measure and Polling, Complementary Information, Runtime model generation etc. are introduced and incorporated to confirm performance enhancement in final trial. Evaluation methods and metrics used here are domain independent and applicable to similar systems.