Towards an Adaptive and Distributed Architecture for Managing Workflow Provenance Data

Workflow provenance data represents the workflow execution behavior, allowing for tracing the generation of the scientific data-flow. Provenance is an important asset to analyze data, identify and handle errors that occurred during the workflow execution through runtime monitoring. The workflow execution engine can also use provenance data to set the initial amount of resources and plan adaptive task scheduling. However, efficiently managing provenance data from distributed workflow execution has several challenges. As the size of workflows increases (in terms of number of activity executions or volume of data to process), the amount of provenance data to be managed also grows, especially in fine grain. Thus, centralized approaches become unviable. In this work we propose an architecture that combines distributed workflow management techniques with distributed provenance data management.