Title :
Efficient Algorithms for Fast Data Transfers Using Long and Large Pipes in WAN Networks
Author :
Schrager, Dan ; Radulescu, Fabian
Author_Institution :
Fac. of Autom. Control & Comput. Sci., Univ. Politeh. of Bucharest, Bucharest, Romania
Abstract :
The pipe programming paradigm is an important Unix innovation which lets processes filter data to obtain a desired output. Extending computation in an Internet distributed environment has led us to design so called long and large pipes which achieve high speed transfer rates between remotely interconnected processes. Our method combines, in a full-fledged client/server application, the use of classical pipes, as a mean of serializing data locally, with multiple parallel TCP connections which yield high bandwidth throughput. The application, called bbftpPRO, brings together the old fashioned techniques of file striping with the new data streaming algorithms at high transfer rates. It includes capabilities for easy traversal of networks protected by firewalls or NAT machines, split-TCP like path support, and last but not least, support for new transmission procedures at variable rate, adapted for multipath implementation at application level. Experimental results have proven the superiority of the proposed data transfer system. Both simple file transfers and arbitrary data streaming were done effectively, efficiently, in parallel, over wide area networks, between pairs of distributed processes connected via extended fast pipes. Our piped approach proved to be portable across heterogeneous systems and most suitable for data intensive computing applications requiring geographically distributed data.
Keywords :
IP networks; Internet; Unix; firewalls; transport protocols; Internet distributed environment; NAT machines; Unix innovation; WAN networks; bbftpPRO; data intensive computing applications; data streaming algorithms; fast data transfer system; filter data processes; firewalls; heterogeneous systems; high bandwidth throughput; high speed transfer rates; local data serialization; multipath implementation; multiple parallel TCP connections; pipe programming paradigm; remotely interconnected processes; split-TCP like path support; transmission procedures; variable rate; wide area networks; Arrays; Bandwidth; Data transfer; Servers; Synchronization; Wide area networks; Writing; HBDP; WAN; multipath; parallel streaming; pipeline;
Conference_Titel :
Control Systems and Computer Science (CSCS), 2013 19th International Conference on
Conference_Location :
Bucharest
Print_ISBN :
978-1-4673-6140-8
DOI :
10.1109/CSCS.2013.6