Evaluation of Test Criteria for Space Application Software Modeling in Statecharts

Several papers have addressed the problem of knowing which software test criteria are better than others with respect to parameters such as cost, efficiency and strength. This paper presents an empirical evaluation in terms of cost and efficiency for one test method for finite state machines, switch cover, and two test criteria of the statechart coverage criteria family, all-transitions and all-simple-paths, for a reactive system of a space application. Mutation analysis was used to evaluate efficiency in terms of killed mutants. The results show that the two criteria and the method presented the same efficiency but all-simple-paths presented a better cost because its test suite is smaller than the one generated by switch cover. Besides, test suite due to the all-simple-paths criterion killed the mutants faster than the other test suites meaning that it might be able to detect faults in the software more quickly than the other criteria.