Multi-objective Test Suite Optimization for Incremental Product Family Testing

The design of an adequate test suite is usually guided by identifying test requirements which should be satisfied by the selected set of test cases. To reduce testing costs, test suite minimization heuristics aim at eliminating redundancy from existing test suites. However, recent test suite minimization approaches lack (1) to handle test suites commonly derived for families of similar software variants under test, and (2) to incorporate fine-grained information concerning cost/profit goals for test case selection. In this paper, we propose a formal framework to optimize test suites designed for sets of software variants under test w.r.t. multiple conflicting cost/profit objectives. The problem representation is independent of the concrete testing methodology. We apply integer linear programming (ILP) to approximate optimal solutions. We further develop an efficient incremental heuristic for deriving a sequence of representative software variants to be tested for approaching optimal profits under reduced costs. We evaluated the algorithm by comparing its outcome to the optimal solution.