A Novel Framework for Monitoring and Analyzing Quality of Data in Simulation Workflows

In recent years scientific workflows have been used for conducting data-intensive and long running simulations. Such simulation workflows have processed and produced different types of data whose quality has a strong influence on the final outcome of simulations. Therefore being able to monitor and analyze quality of this data during workflow execution is of paramount importance, as detection of quality problems will enable us to control the execution of simulations efficiently. Unfortunately, existing scientific workflow execution systems do not support the monitoring and analysis of quality of data for multi-scale or multi-domain simulations. In this paper, we examine how quality of data can be comprehensively measured within workflows and how the measured quality can be used to control and adapt running workflows. We present a quality of data measurement process and describe a quality of data monitoring and analysis framework that integrates this measurement process into a workflow management system.