An Online Provenance Service for Distributed Metabolic Flux Analysis Workflows

Scientific applications in the area of Systems Biology become more and more demanding in terms of workflow organization and required computational resources. Metabolic flux analysis with carbon tracer experiments (¹³C-MFA) is a particularly challenging technology consisting of several tightly interwoven building blocks, some of which are quite compute-intensive while others may require human expert interaction. Within these buildings blocks and at their interfaces, large amounts of provenance data (i.e., process messages, logs and metadata) are created. An online provenance approach for a scientific workflow framework is presented to efficiently monitor and analyze long running parallel ¹³C-MFA applications. The design of our approach consists of a cross-language programming interface for generating messages, a provenance store for collecting log messages from distinct workflows, and a versatile user client for querying and filtering data of interest. Seamless integration into the BPEL-based scientific workflow framework is enabled by web service interfaces. Salient features of our provenance solution are interactivity and scalability, as demonstrated by two examples. The framework provides the capability to capture, filter, query, and trace provenance data on demand. Our solution leverages scientific analysis of complex ¹³C-MFA workflows.