An experiment on lead-time impact in testing of distributed real-time systems

In the search for an efficient test method for distributed real-time systems, an experiment is conducted that investigates the impact on the lead time required to identify a defect, depending on what information is available. One group of experiment participants were given a system monitoring trace, while a control group were given node monitoring traces. The information made available in the two sets of traces was identical except for which monitoring approach was being used. The system under study is the control software for a private branch exchange (PBX) with three nodes. The experiment, conducted with 23 participants, suggests with statistically significant confidence that it takes a shorter time to isolate a defect using system monitoring traces than using node monitoring traces. Further, the identification of the defect source is more correct with system monitoring. These findings help in motivating investments in technology that produces system monitoring traces