Empirical Studies on Test Effectiveness for Database Applications

This paper presents empirical studies of test effectiveness for Java database applications. We primarily address two research questions: (1) What is the relationship between test set size, coverage criteria, and test effectiveness for Java/JDBC programs? (2) Which adequacy criterion is more suitable for testing Java database applications, statement coverage, branch coverage, or command form coverage? We collected data sets by experimenting on several moderately complex open source database application programs, and performed a series of rigorous and comprehensive statistical analyses. The results show that linear models fit the relationship of size and coverage to effectiveness well and that coverage appears to have greater impact on effectiveness than size. However, surprisingly, command form coverage was not consistently more effective than statement or branch coverage.