Beyond the storage area network: data intensive computing in a distributed environment

NASA Earth and space science applications are currently utilizing geographically distributed computational platforms. These codes typically require the most compute cycles and generate the largest amount of data over any other applications currently supported by NASA. Furthermore, with the development of a leadership class SGI system at NASA Ames (Project Columbia), NASA has created an agency wide computational resource. This resource is heavily employed by Earth and space science users resulting in large amounts of data. The management of this data in a distributed environment requires a significant amount of effort from the users. This paper defines the approach taken to create an enabling infrastructure to help users easily access and move data across distributed computational resources. Specifically, this paper discusses the approach taken to create a wide area storage area network (SAN) using the SGI CXFS file system over standard TCP/IP. In addition, an emerging technology test bed initiative is described to show how NASA is creating an environment to continually evaluate new technology for data intensive computing.