On the performance of MapReduce: A stochastic approach

Author

Ahmed, Syed Thouheed ; Loguinov, Dmitri

Author_Institution

Texas A&M Univ., College Station, TX, USA

fYear

2014

fDate

27-30 Oct. 2014

Firstpage

49

Lastpage

54

Abstract

MapReduce is a highly acclaimed programming paradigm for large-scale information processing. However, there is no accurate model in the literature that can precisely forecast its run-time and resource usage for a given workload. In this paper, we derive analytic models for shared-memory MapReduce computations, in which the run-time and disk I/O are expressed as functions of the workload properties, hardware configuration, and algorithms used. We then compare these models against trace-driven simulations using our high-performance MapReduce implementation.

Keywords

Big Data; parallel processing; shared memory systems; sorting; stochastic processes; Big Data; MapReduce performance; analytic models; disk I/O; external sort; hardware configuration; high-performance MapReduce implementation; large-scale information processing; programming paradigm; resource usage; run-time; shared-memory MapReduce computations; stochastic approach; trace-driven simulations; workload properties; Analytical models; Arrays; Computational modeling; Merging; Programming; Random access memory; Sorting; Big Data; Disk I/O; External Sort; MapReduce;

fLanguage

English

Publisher

ieee

Conference_Titel

Big Data (Big Data), 2014 IEEE International Conference on

Conference_Location

Washington, DC

Type

conf

DOI

10.1109/BigData.2014.7004212

Filename

7004212