Mining Scientific Data using the Internet as the Computer

This paper describes approaches and methodologies facilitating the analysis of large amounts of distributed scientific data. The existence of full-featured analysis tools, such as the Algorithm Development and Mining (ADaM) toolkit and online data repositories now provide easy access and analysis capabilities to large amounts of data. However, there are obstacles to getting the analysis tools and the data together in a workable environment. Does one bring the data to the tools or deploy the tools close to the data? The large size of many current Earth science datasets incurs significant overhead in network transfer for analysis workflows, even with the current advanced networking capabilities. We are developing two solutions for this problem that address different analysis scenarios. The first is a Data Center Deployment of the analysis services for large data selections, orchestrated by a remotely defined analysis workflow. The second is a Data Mining Center approach of providing a cohesive analysis solution for smaller subsets of data. The two approaches can be complementary and thus provide flexibility for researchers to exploit the best solution for their data requirements.