Extracting information ASAP!

Designing I/O systems capable of scaling up to deal with the next generation of extreme scale scientific environments is a significant challenge. Scientific applications already strain the capabilities of current filesystems and storage systems. This work presents a middleware-based approach that generalizes from previous work on staging areas to focus more generally on staging resources. Exploiting the steady increase of the ratio of compute capability to I/O bandwidth, the EnStage middleware system allows for metadata characterization and I/O processing to occur when as where appropriate. This includes in reserved staging areas, buffered memory, and even in the writing processes´ execution context. Using the EnStage extension to previous work, we find a 1.4% increase in runtime due to additional functionality resulted in I/O time in the staging area dropping to only 16% of the non-reduced output.