Transformation Rules for Platform Independent Testing: An Empirical Study

Most Model-Driven Development projects focus on model-level functional testing. However, our recent study found an average of 67% additional logic-based test requirements from the code compared to the design model. The fact that full coverage at the design model level does not guarantee full coverage at the code level indicates that there are semantic behaviors in the model that model-based tests might miss, e.g., conditional behaviors that are not explicitly expressed as predicates and therefore not tested by logic-based coverage criteria. Avionics standards require that the structure of safety critical software is covered according to logic-based coverage criteria, including MCDC for the highest safety level. However, the standards also require that each test must be derived from the requirements. This combination makes designing tests hard, time consuming and expensive to design. This paper defines a new model that uses transformation rules to help testers define tests at the platform independent model level. The transformation rules have been applied to six large avionic applications. The results show that the new model reduced the difference between model and code with respect to the number of additional test requirements from an average of 67% to 0% in most cases and less than 1% for all applications.