Decision Support for Test Management and Scope Selection in a Software Product Line Context

In large software organizations with a product line development approach, system test planning and scope selection is a complex tasks for which tool support is needed. Due to repeated testing: across different testing levels, over time (test for regression) as well as of different variants, the risk of double testing is large as well as the risk of overlooking important tests, hidden by the huge amount of possible tests. This paper discusses the need and challenges of providing decision support for test planning and test selection in a product line context, and highlights possible paths towards a pragmatic implementation of context-specific decision support of various levels of automation. With existing regression testing approaches it is possible to provide automated decision support in a few specific cases, while test management in general may be supported through visualization of test execution coverage, the testing space and the delta between the sufficiently tested system and the system under test. A better understanding of the real world context and how to map research results to the same is needed.