An experience with measuring multi-user online task performance

Growing demand for usability testing as a part of iterative design process requires novel approaches and methods to accelerate the process. This paper introduces a new approach of conducting MUST (Multiple-User Simultaneous Testing) using the Autopilot feature of Morae as a potential way for discounted usability testing method. Seventeen participants performed 15 information retrieval tasks on an academic website using MUST. Taking proper steps in research design and study set up when collecting data on online user interaction in a group setting can improve data quality by avoiding unnecessary mistakes. Also, knowledge about advanced features available in the data collection software can help information retrieval and usability evaluation researchers to become well-prepared for the challenges of data collection. Automatic reports generated by Autopilot are provided that include graphs of average time on tasks and participant responses on likert scale. Advantages and disadvantages of the Autopilot feature are also shared.