Spoken language interaction in a goal-directed task

To study the spoken language interface in the context of a complex problem-solving task, a group of users are asked to perform a spreadsheet task, alternating voice and keyboard input. A total of 40 tasks are performed by each participant, the first 30 in a group (over several days), the remaining ones a month later. The voice spreadsheet program is extensively instrumented to provide detailed information about the components of the interaction. These data, as well as analysis of the participant´s utterances and recognizer output, provide a fairly detailed picture of spoken language interaction. Although task completion by voice takes longer than by keyboard, analysis shows that users would be able to perform the spreadsheet task faster by voice, if two key criteria could be met: recognition occurs in real-time, and the error rate is sufficiently low. This initial experience with a spoken language system also allows the identification of several metrics, beyond those traditionally associated with speech recognition, that can be used to characterize system performance