Analysis of faults in an N-version software experiment

The authors have conducted a large-scale experiment in N-version programming. A total of 27 versions of a program were prepared independently from the same specification at two universities. The results of executing the versions revealed that the versions were individually extremely reliable but that the number of input cases in which more than one failed was substantially more than would be expected if they were statistically independent. After the versions had been executed, the failures of each version were examined and the associated faults located. It appears that minor differences in the software development environment would not have a major impact in reducing the incidence of faults that cause correlated failures