A generic method for statistical testing

This paper addresses the problem of selecting finite test sets and automating this selection. Among these methods, some are deterministic and some are statistical. The kind of statistical testing we consider has been inspired by the work of Thevenod-Fosse and Waeselynck. There, the choice of the distribution on the input domain is guided by the structure of the program or the form of its specification. In the present paper, we describe a new generic method for performing statistical testing according to any given graphical description of the behavior of the system under test. This method can be fully automated. Its main originality is that it exploits recent results and tools in combinatorics, precisely in the area of random generation of combinatorial structures. Uniform random generation routines are used for drawing paths from the set of execution paths or traces of the system under test. Then a constraint resolution step is performed, aiming to design a set of test data that activate the generated paths. This approach applies to a number of classical coverage criteria. Moreover, we show how linear programming techniques may help to improve the quality of test, i.e. the probabilities for the elements to be covered by the test process. The paper presents the method in its generality. Then, in the last section, experimental results on applying it to structural statistical software testing are reported.