A model for performance evaluation of interactive systems

We describe a quantitative model for the performance evaluation of interactive computer systems. The approach involves the development of an “interaction graph” or state transition diagram to describe the user-machine interaction. Given numerical data on transition times and probabilities, the model can be used to perform sensitivity analyses of changes in system parameters and user behavior. To illustrate the model, we use empirical data from field and laboratory experiments designed to compare a prototype natural language query system with a formal (relational) query system. The general approach is applicable in a broad range of other contexts including bibliographic retrieval and the analysis of web-log data. It should be of interest to both system developers and potential users of these systems