Comparing Two Black-Box Testing Strategies for Software Product Lines

SPL testing has been considered a challenging task,mainly due to the diversity of products that might be generatedfrom an SPL. To deal with this problem, some techniques for derivingproduct specific functional test cases have been proposed.However, this research area still lacks empirical studies showingthe benefits of using such techniques. This paper presents a studythat empirically compares two different black-box manual testdesign techniques: a generic technique that we have observed inan industrial test execution environment and a product specifictechnique whose functional test cases could be derived using anySPL technique that considers variations in functional tests. Weevaluated their impact from the point of view of the test execution process, achieving results that indicate that executing product specific test cases is faster and generates fewer errors.