Iterative Instrumentation for Code Coverage in Time-Sensitive Systems

In software testing, runtime code coverage is usually measured by instrumenting the executable code. In most cases the effect of the additional instructions is negligible, but for time-sensitive systems it can potentially alter the timing of executing code. This may lead to inconsistent test results, as the tests behave differently when run against non-instrumented code. In this paper, we discuss traditional code coverage techniques (i.e. instrumentation) and our code coverage technique called "iterative instrumentation" which has no runtime overhead. We analyze the impact of instrumentation runtime overhead through a case study of heuristic pathfinders. Next, we compare the effectiveness of iterative instrumentation to traditional instrumentation for measuring code coverage on a case study of control software. Finally, we discuss possible techniques to improve the quality of our technique including the limitations of this paper. Our studies confirm that instrumentation runtime overhead can alter the timing of time-sensitive software systems while our technique can be used effectively with no runtime overhead.