A New Metric for Measuring Metamodels Quality-of-Fit for Deterministic Simulations

Metamodels are used to provide simpler prediction means than the complex simulation models they approximate. Accuracy of a metamodel is one fundamental criterion that is used as the basis for accepting or rejecting a metamodel. Average-based metrics such as root-mean-square error RMSE and R-square are often used. Like all other average-based statistics, these measures are sensitive to sample sizes unless the number of test points in these samples is adequate. A new metric that can be used to measure metamodels fit quality, called metamodel acceptability score MAS, is introduced. The proposed metric gives readily interpretable meaning to metamodels acceptability. Furthermore, initial studies show that MAS is less sensitive to test sample sizes compared to average-based validation measures