A methodological adaptation for heuristic evaluation of HRI

Usability inspection methods like cognitive walkthroughs and guideline-based evaluations of robotic systems find more and more usage in the design of human-robot interaction. However, due to the fact that robots can be used in a variety of different scenarios, applying these methods to robots is rather difficult. Expert evaluation methods like heuristic evaluations thus need a methodological adaptation to be reasonably applied for human-robot interaction scenarios. We propose a methodological adaptation of remote heuristic evaluation based on video scenarios. To investigate this methodological adaptation we conducted a heuristic evaluation case study based on six different video scenarios with three different robots. Traditional heuristic evaluation data was gathered (number of found usability problems and severity rankings of the problems) and additionally a feedback questionnaire on the method itself was distributed to the participating expert evaluators. We could show that video-based remote heuristic evaluations conducted by interdisciplinary expert evaluators are a valuable approach to detect usability problems in human-robot interaction design in an early development phase.