Data composition patterns in service-based workflows

Workflow engines are powerful tools to implement data-intensive scientific applications exploiting parallel grid resources transparently. We discuss the advantages of implementing applications as workflows of services when dealing with large data sets. We show how the graph of services associated with data composition operators enable the description of complex data flows in a very compact format. We define a strict semantics for two common composition operators and we propose an algorithm to consistently satisfy this semantics all along the workflow execution. Finally, we show how our approach enables parallel execution of the application while preserving the data flow semantics. We implemented the algorithm proposed in MOTEUR, an open source workflow engine designed to execute parallel and data-intensive applications.