Estimators for fault tolerance coverage evaluation

The problem of estimating the coverage of a fault tolerance mechanism through statistical processing of observations collected in fault injection experiments is addressed. A formal definition of coverage is given in terms of the fault and activation sets that characterize the input space. Two categories of sampling techniques are considered for coverage estimation: sampling in the whole space and sampling in a space partitioned into classes. The estimators for each technique are compared by means of hypothetical examples. Techniques for early estimations of coverage are then studied. These techniques allow unbiased estimations of coverage to be made before all classes of the sampling space have been tested. Finally, the "no-reply" problem that hampers most practical fault-injection experiments is discussed and an a posteriori stratification technique is proposed that allows the scope of incomplete tests to be widened by accounting for available structural information about the target system.