Data Consolidation: A Task Scheduling and Data Migration Technique for Grid Networks

In this work we examine a task scheduling and data migration problem for grid networks, which we refer to as the data consolidation (DC) problem. DC arises when a task needs for its execution two or more pieces of data, possibly scattered throughout the grid network. In such a case, the scheduler and the data manager must select the data replicas to be used and the site where these will accumulate for the task to be executed. The policies for selecting the data replicas and the data consolidating site comprise the data consolidation problem. We propose and experimentally evaluate a number of DC techniques. Our simulation results brace our belief that DC is an important technique for data grids since it can substantially improve task delay, network load and other performance related parameters.