DocumentCode
3529689
Title
Horde: A parallel programming framework for clusters
Author
Xu, Wenhao ; Wu, Yongwei ; Xue, Wei ; Zhang, Wusheng ; Yuan, Ye ; Zhang, Kai
Author_Institution
Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China
fYear
2009
fDate
23-24 Aug. 2009
Firstpage
96
Lastpage
101
Abstract
Horde is a general programming framework for writing parallel applications in clusters. A computing task is modeled as a graph in Horde. Each sub-task maps to one vertex and data channels map to edges in the graph. Programming with Horde is very simple by writing sequential code for vertexes and adding edges to link vertexes. Horde can tolerant transient fault and provide support to write code for toleranting permanent faults. Horde is portable and support various cluster job managers. We evaluate Horde´s efficiency in communication through micro benchmarks and prove the easy-of-use of Horde by implementing a MapReuce engine. The test in a small scale cluster show that our implementation outperforms Hadoop.
Keywords
fault tolerant computing; graph theory; parallel programming; workstation clusters; Horde parallel programming; cluster job management; data channel; graph theory; sequential code; transient fault tolerance; Application software; Benchmark testing; Computer science; Concurrent computing; Engines; High performance computing; Libraries; Parallel processing; Parallel programming; Writing; Cluster Computing; Concurrency; Parallel Programming; Task Graph;
fLanguage
English
Publisher
ieee
Conference_Titel
Web Society, 2009. SWS '09. 1st IEEE Symposium on
Conference_Location
Lanzhou
Print_ISBN
978-1-4244-4157-0
Electronic_ISBN
978-1-4244-4158-7
Type
conf
DOI
10.1109/SWS.2009.5271793
Filename
5271793
Link To Document