A Diagnostic Point of View for the Optimization of Preparation Costs in Runtime Testing

Runtime testing is emerging as the solution for the validation and acceptance testing of service-oriented systems, where many services are external to the organization, and duplicating the system´s components and their context is too complex, if possible at all. In order to perform runtime tests, an additional expense in the test preparation phase is required, both in software development and in hardware. Preparation cost prioritization methods have been based on runtime testability (i.e, coverage) and do not consider whether a good runtime testability is sufficient for a good runtime diagnosis quality in case faults are detected, and whether this diagnosis will be obtained efficiently (i.e., with a low number of test cases). In this paper we show (1) the direct relationship between testability and diagnosis quality, that (2) these two properties do not guarantee an efficient diagnosis, and (3) a measurement that ensures better prediction of efficiency.