An Anticipative Recursively-Adjusting Mechanism for redundant parallel file transfer in data grids

In data grids, co-allocation architectures were developed to enable parallel downloads of datasets from selected replica servers. As Internet is usually the underlying network of a grid, network bandwidth plays as the key factor affecting file transfers between clients and servers. In this paradigm, there are still some challenges which need to be solved, such as to reduce differences in finish times between selected replica servers; to avoid traffic congestion resulting from transferring the same blocks in different links among servers and clients; and to manage network performance variations among parallel transfers. In this paper, we propose the anticipative recursively-adjusting mechanism (ARAM) scheme to adjust the workloads on selected replica servers and handle unpredictable variations in network performance by those servers. Our algorithm is based on using the finish rates of previously assigned transfers to anticipate the bandwidth status for next section in order to adjust workloads, and to reduce file transfer times in grid environments. Our approach is useful in grid environments with unstable network link. It not only reduces idle time wasted waiting for the slowest server, but also decreases file transfer completion times.