A hypothesis testing framework for evaluating ecosystem model performance

Model evaluation is argued to necessitate the use of a hypothesis testing framework instead of the use of goodness-of-fit (GOD) against time series data. A test statistic T is developed, based on model deviation from expected system behaviors. If a model does not exceed the error bounds on expected behavior, then we cannot say that it differs from the real system. Precision is measured not by degree of fit to a set of data but by the precision (width of confidence limits around) the expected system behavior. Implementation of this approach is enhanced by development of new test criteria that evaluate biological and ecological realism based on aggregate indices, failure modes, extreme condition tests, and others. Model performance is assessed by the extent to which the model falls within these expected bounds. Using this test statistic as a basis, revised approaches to sensitivity and uncertainty analysis are recommended. Factors not addressed by these techniques require structural analysis, which is based on comparisons between models with different structures. The application of the advocated approach should enhance confidence in ecosystem models applied to policy issues such as natural resource management or global change impacts.