Evolutionary White-Box Software Test with the EvoTest Framework: A Progress Report

Evolutionary white-box software testing has been extensively researched but is not yet applied in industry. In order to investigate the reasons for this, we evaluated a prototype version of a tool, representing the state-of-the-art for evolutionary structural testing, which is targeted at industrial use. The focus was on the applicability of the structural test tool in the industrial context and not on assessment of the test cases generated. Four case studies, each consisting of an embedded software module from the automotive industry implemented in the C language, were evaluated with the tool. The case studies had to be customized to cope with the limitations of the tool and in all, test case generation succeeded for 37% of the functions selected for the evaluation. Weaknesses of the tool were reported to the developers and subsequently eliminated, resulting in a later version of the tool being able to process 82% of the selected case study functions. However, the study shows that significant engineering work is still required before evolutionary structural testing is ready for industrial application.