Minimising the Preparation Cost of Runtime Testing Based on Testability Metrics

Test cost minimisation approaches have traditionally been devoted to minimising "execution costs\´\´, while maximising coverage or reliability. However, in a runtime testing context, the amount of coverage or reliability that can be achieved, in other words, the system\´s Runtime Testability, is limited by the adverse effects that the interferences of runtime tests have on the system. Supporting runtime testing, therefore, introduces an additional cost in "preparatory\´\´ activities in software, (e.g., testable components) and in hardware (e.g., more memory), before certain runtime tests can be executed. In this paper we present a low-complexity, cost minimisation algorithm for the optimal selection of preparation activities, based on a near-optimal trade-off between preparation cost and a structure-based measurement of Runtime Testability, coined the Runtime Testability Metric (RTM). We perform a theoretical and empirical validation of RTM, showing that RTM is indeed a valid, and reasonably accurate measurement with ratio scale. We also present empirical data demonstrating the near-optimal performance at a low computational cost of our algorithm.