CometCloud: Enabling Software-Defined Federations for End-to-End Application Workflows

Emerging applications, from big science to the Internet of Things, increasingly involve dynamic and data-driven end-to-end workflows with large and often heterogeneous requirements. These applications require platforms that dynamically and flexibly combine resources across systems and data centers--for example, to aggregate capacity or capabilities, or integrate data stores when moving data is no longer an option. The CometCloud project at the Rutgers Discovery Informatics Institute (RDI2) aims to provide infrastructure and programming support for enabling such end-to-end application workflows. CometCloud enables flexible software-defined synthesis of custom cyberinfrastructure through the autonomic, on-demand federation of geographically distributed compute and data resources. CometCloud exposes the federated cyberinfrastructure using elastic cloud abstractions and science-as-a-service platforms. The authors introduce CometCloud, use sample application use cases to illustrate its operation, and present current and future research directions toward supporting pervasive application workflows enabled by the Internet of Things.