An adaptive middleware framework for Scientific Computing at extreme scales

Large computing systems including clusters, clouds, and grids, provide high-performance capabilities that can be utilized for scientific applications. As the ubiquity of these systems increases and the scope of analysis performed on them expand, there is a growing need for applications that do not require users to learn the details of high-performance computing, and are flexible and adaptive to accommodate the best time-to-solution. In this paper we introduce a new adaptive capability for the MeDICi middleware and describe the applicability of this design to a scientific workflow application for biology. This adaptive framework provides a programming model for implementing a workflow using high-performance systems and enables the compute capabilities at one site to automatically analyze data being generated at another site. This adaptive design improves overall time-to-solution by moving the data analysis task to the most appropriate resource dynamically, automatically reacting to failures and load fluctuations.