DocumentCode
2787349
Title
A Hierarchical Approach to Maximizing MapReduce Efficiency
Author
Xiao, Zhiwei ; Chen, Haibo ; Zang, Binyu
Author_Institution
Parallel Process. Inst., Fudan Univ., Shanghai, China
fYear
2011
fDate
10-14 Oct. 2011
Firstpage
167
Lastpage
168
Abstract
In this paper, we argued that Hadoop has limitations in exploiting data locality and task parallelism for multi-core platforms. We then extended Hadoop with a hierarchical MapReduce scheme. An in-memory cache scheme is also seamlessly integrated to cache data that is likely to be accessed in memory. Evaluation showed that the hierarchical scheme outperforms Hadoop ranging from 1.4x to 3.5x.
Keywords
cache storage; multiprocessing systems; parallel processing; Hadoop; MapReduce efficiency; data locality; hierarchical MapReduce scheme; in-memory cache scheme; multicore platform; task parallelism; Distance measurement; Memory management; Multicore processing; Parallel processing; Protocols; Runtime; Servers; MapReduce; hierarchical approach; performance;
fLanguage
English
Publisher
ieee
Conference_Titel
Parallel Architectures and Compilation Techniques (PACT), 2011 International Conference on
Conference_Location
Galveston, TX
ISSN
1089-795X
Print_ISBN
978-1-4577-1794-9
Type
conf
DOI
10.1109/PACT.2011.22
Filename
6113798
Link To Document