Toward a model-driven engineering framework for reproducible simulation experiment lifecycle management

Goal-directed reproducible experimentation with simulation models is still a significant challenge. The underutilization of design of experiments, limited transparency in the collection and analysis of results, and ad-hoc adaptation of experiments as learning takes place continue to hamper reproducibility and hence cause a credibility gap. In this study, we propose a strategy that leverages the synergies between model-driven engineering, intelligent agent technology, and variability modeling to support the management of the lifecycle of a simulation experiment. Experiment design and workflow models are introduced for configurable experiment synthesis and execution. Feature-based variability modeling is used to design a family of experiments, which can be leveraged by ontology-driven software agents to configure, execute, and reproduce experiments. Online experiment adaptation is proposed as a strategy to facilitate dynamic experiment model updating as objectives shift from validation to variable screening, understanding, and optimization.