A Guide to Scientific Evaluation in Information Visualization

This paper addresses some fundamental and practical issues that should be considered when pursuing evaluation studies in Information Visualization. The main focus is on quantitative experimental research but the general information applies to all kinds of studies. The purpose is to increase awareness of what constitutes a sound scientific approach to evaluation and to point out common pitfalls and mistakes during the phases of such study. These phases cover how to plan, design, conduct and analyse the outcome of an evaluation and finally how to report in a way that enhances readability, provides details relevant to the outcome and that allows replication. The paper could be used as a guide when conducting evaluation and it could also be helpful when reviewing publications since the same rules apply.