ECOVAL: A Framework for Increasing the Ecological Validity in Usability Testing

The term ecological validity refers to the study of the measurable correspondence between experimental design and ecological setting (i.e., Representative users performing real tasks in their natural environment). Therefore, it is important to anyone intending to generalize the conclusions from the particular circumstances of an experiment to wider ecological situations. However, there is no operational definition that enables to objectively quantify the ecological validity of an experimental design. The ECOVAL framework offers a formal starting point to bridge this gap by helping to both define and increase the ecological validity in usability testing. A methodology, a tool and guidelines to increase ecological validity are presented in an industrial case study. The return on investment of increasing ecological validity is examined in terms of user efficiencies and usability practice. Finally, recommendations to increase ecological validity are introduced, contributions discussed, and outlines for future work presented.