Title :
On the Feasibility of Byzantine Fault-Tolerant MapReduce in Clouds-of-Clouds
Author :
Correia, Miguel ; Costa, Pyramo ; Pasin, Marco ; Bessani, Alysson ; Ramos, Felix ; Verissimo, P.
Author_Institution :
INESC-ID/IST, Lisbon, Portugal
Abstract :
MapReduce is a framework for processing large data sets largely used in cloud computing. MapReduce implementations like Hadoop can tolerate crashes and file corruptions, but there is evidence that general arbitrary faults do occur and can affect the correctness of job executions. Furthermore, many individual cloud outages have been reported, raising concerns about depending on a single cloud. We present a MapReduce runtime that tolerates arbitrary faults and runs in a set of clouds at a reasonable cost in terms of computation and execution time. The main challenge is to avoid sending through the internet the huge amount of data that would normally be exchanged between map and reduce tasks.
Keywords :
cloud computing; data handling; fault tolerant computing; Byzantine fault-tolerant MapReduce; Hadoop; Internet; MapReduce runtime; arbitrary fault tolerance; cloud computing; cloud outage; clouds-of-clouds; crash tolerance; file corruption; job execution correctness; large data set processing; Cloud computing; Fault tolerant systems; Logic gates; Redundancy; Servers;
Conference_Titel :
Reliable Distributed Systems (SRDS), 2012 IEEE 31st Symposium on
Conference_Location :
Irvine, CA
Print_ISBN :
978-1-4673-2397-0
DOI :
10.1109/SRDS.2012.46