On the Relative Strengths of Model-Based and Dynamic Event Extraction-Based GUI Testing Techniques: An Empirical Study

Many software systems rely on graphical-user interfaces (GUIs) to support user interactions. The correctness of these GUIs affects the overall quality of the systems, and thus, it is important that GUIs be tested. To support such testing, GUI test case generation techniques based on graph models such as event flow graphs (EFGs) have been used to generate test cases in the form of sequences of events. Models, however, are abstractions of dynamic behavior and may not accurately reflect actual system behavior, thus, test case generation techniques based on models may create nonexecutable test cases and miss important events. Test case generation techniques based on dynamic event extraction-based approaches, in contrast, may suffer less from these effects. As a consequence, we expect that the two approaches will differ in terms of cost and effectiveness. In this paper, we report the results of an empirical study comparing the cost and effectiveness of model-based and dynamic event extraction-based test case generation approaches. Our results show that event extraction-based approaches, though more expensive than model-based approaches, are indeed more effective in terms of their ability to achieve code coverage and to cover events, but at additional cost, with implications for both researchers and practitioners.