Comparisons of transport and dispersion model predictions of the european tracer experiment: area- and population-based user-oriented measures of effectiveness

In October 1994, a tracer gas was released from a location in northwestern France and tracked at 168 sampling locations in 17 countries across Europe. This release, known as the European Tracer Experiment (ETEX), resulted in the collection of a wealth of data. This paper applies a previously described user-oriented measure of effectiveness (MOE) methodology to evaluate the predictions of 46 models against the long-range ETEX observations. The paper extends previous work by computing MOE values that are based on “true” areas (e.g., in square kilometers) and on actual European population distributions. In this way, assessments of model predictions of ETEX are placed in a possible operational context. The predictive performance of the models were assessed with nominal, area-, dosage-, and population-based MOE values and ranked using a few notional user-based scoring functions. This study finds that the rankings of some models—in particular, several of the model predictions that were ranked in the top 10—are relatively insensitive to the particular MOE technique used. This robust behavior, with respect to analysis assumptions, is regarded as an important feature of model performance.