An architecture and implementation to support large-scale data access in scientific simulation environments

At Sandia National Laboratories, a Data Services system has been developed to provide web-based access to high-performance computing clusters that host a set of post-processing applications for very large-scale data manipulation and visualization. A three-tier architecture provides a meta-framework for a collection of smaller frameworks, each of which satisfies a particular aspect of the overall system, including frameworks for a common data model, distributed resource management, component-based software on the cluster, and security. A prototype implementation has been completed which demonstrates the use of all of these frameworks in an integrated environment to provide end users the ability to manage and understand simulation results for very large, complex problems.