System test cost modelling based on event rate analysis

Few methods are available which support the development of a cost-optimised system test strategy. One reason for this is that system complexity inhibits the objective comparison of alternative approaches. This paper describes a parameterised model which is used to represent system behaviour during both production testing and early field run-time. The model represents the occurrence rate of error and failure events under test or application workloads and the model parameters directly characterise system test effectiveness. Event rate models describing the production testing and early field run-time are incorporated into a cost model. The basis of the cost model is a cost function which includes the set-up and running costs of both environments. The results identify the relationship between the overall “cost of test” and some of the key variables in the cost function. The approach can be applied to both hardware and software related events thus permitting a more accurate representation of the customer view of system quality