The Systems Approach to Test Evaluation

The evolution of large and sophisticated systems has created a requirement for extensive system tests programs. The argument that the change in scale of testing affects organizational communications so as to decrease the visibility provided by traditional management tools is presented. To attack this problem, a recommendation is made to establish a central test evaluation group with the primary goal of feeding back intelligence from the test program to the designers and managers. The concept of test "intelligence" is defined and a typical activity flow for the evaluation group is described. The general factors which determine the value of information are related to the specific variables of a test program. A rationale is developed for planning an evaluation approach based on the tradeoff between time invested in evaluation and the change in the value of the evaluation product (intelligence) as it ages. A functional approach to the reporting scheme is given. Report contents and timing are keyed to the program decision requirements. The potential benefits of a central, independent, and objective test evaluation are suggested for the designer, the manager, and the customer.