Analysis of Mistakes as a Method to Improve Test Case Design

Test Design -- how test specifications and test cases are created -- inherently determines the success of testing. However, test design techniques are not always properly applied, leading to poor testing. We have developed an analysis method based on identifying mistakes made when designing the test cases. Using an extended test case template and an expert review, the method provides a systematic categorization of mistakes in the test design. The detailed categorization of mistakes provides a basis for improvement of the Test Case Design, resulting in better tests. In developing our method we have investigated over 500 test cases created by novice testers. In a comparison with industrial test cases we could confirm that many of these mistake categories remain relevant also in an industrial context. Our contribution is a new method to improve the effectiveness of test case construction through proper application of test design techniques, leading to an improved coverage without loss of efficiency.