Title :
MCD: Overcoming the Data Download Bottleneck in Data Centers
Author :
Kaiser, Jasmin ; Meister, Dirk ; Gottfried, Viktor ; Brinkmann, Andre
Author_Institution :
Johannes Gutenberg Univ., Mainz, Germany
Abstract :
The data download problem in data centers describes the increasingly common task of coordinated loading of identical data to a large number of nodes. Data download is seen as a significant problem in exascale HPC applications. Uncoor-dinated reading from a central file server creates contention at the file server and its network interconnect. We propose and evaluation a reliable multicast based approach to solve the data download problem. The MCD system builds a logical multi-rooted tree based on the physical network topology and uses the logical view for a two-phase approach. In the first phase, the data is multicasted to all nodes. In the second phase, the logical tree is used for an efficient error-correction. We evaluate the approach against the Twitter´s Murder, which is BitTorrent-based data download solution used to deploy code binaries to thousands of nodes. The evaluation features a simulation of up to 10,000 nodes and shows that MCD finishes the reliable data download significantly faster. The simulation results are finally validated using a real-world deployment of more than 100 nodes.
Keywords :
computer centres; file servers; parallel processing; BitTorrent-based data download solution; MCD system; Twitter Murder solution; central file server; code binary; coordinated data loading; data centers; data download bottleneck; error correction; exascale HPC applications; high performance computing; logical multirooted tree; logical view; network interconnect; physical network topology; reliable multicast based approach; two-phase approach; Maintenance engineering; Network topology; Packet loss; Protocols; Receivers; Servers;
Conference_Titel :
Networking, Architecture and Storage (NAS), 2013 IEEE Eighth International Conference on
Conference_Location :
Xi´an
DOI :
10.1109/NAS.2013.18