Title :
Foreseer: Workload-Aware Data Storage for MapReduce
Author :
Jia Zou ; Juwei Shi ; Tongping Liu ; Zhao Cao ; Chen Wang
fDate :
June 29 2015-July 2 2015
Abstract :
Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads´ data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.
Keywords :
data handling; parallel processing; storage management; Foreseer; MapReduce auto-tuning techniques; data placement parameters; enterprise production systems; inter-job WORM scenario; inter-job write once read many scenario; online cross-layer solution; workload data access information prediction; workload-aware data storage; Clustering algorithms; Distributed databases; Greedy algorithms; Grippers; Optimization; Partitioning algorithms; Writing;
Conference_Titel :
Distributed Computing Systems (ICDCS), 2015 IEEE 35th International Conference on
Conference_Location :
Columbus, OH
DOI :
10.1109/ICDCS.2015.89
