Comparison of analysis strategies for screening designs in large-scale computer simulation models

In large-scale computer simulation models it is often necessary to perform a screening experiment to reduce the number of factors to be examined in subsequent analysis. This study evaluated the results of a Plackett-Burman screening design using three different analysis strategies: 1) an approach due to Box and Meyer (1993); 2) an approach due to Hamada and Wu (1992); and 3) a standard Response Surface Methodology (RSM) approach. These strategies or methodologies were used to identify the active/significant factors across 17 different model outputs. The results from these three methodologies were then compared against each other for any notable differences in the identified significant factors. In one instance, where there was a notable difference, further analysis was performed in an attempt to ascertain which methodology was the best predictor for that specific response. A Resolution V design was used in this subsequent analysis to produce a validation model, which was then used to compare the three initial analysis strategies. The strategy/methodology that produced the model with the smallest mean absolute percent error (MAPE), the measurement criteria, was selected as the best for that response.