• DocumentCode
    3529689
  • Title

    Horde: A parallel programming framework for clusters

  • Author

    Xu, Wenhao ; Wu, Yongwei ; Xue, Wei ; Zhang, Wusheng ; Yuan, Ye ; Zhang, Kai

  • Author_Institution
    Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China
  • fYear
    2009
  • fDate
    23-24 Aug. 2009
  • Firstpage
    96
  • Lastpage
    101
  • Abstract
    Horde is a general programming framework for writing parallel applications in clusters. A computing task is modeled as a graph in Horde. Each sub-task maps to one vertex and data channels map to edges in the graph. Programming with Horde is very simple by writing sequential code for vertexes and adding edges to link vertexes. Horde can tolerant transient fault and provide support to write code for toleranting permanent faults. Horde is portable and support various cluster job managers. We evaluate Horde´s efficiency in communication through micro benchmarks and prove the easy-of-use of Horde by implementing a MapReuce engine. The test in a small scale cluster show that our implementation outperforms Hadoop.
  • Keywords
    fault tolerant computing; graph theory; parallel programming; workstation clusters; Horde parallel programming; cluster job management; data channel; graph theory; sequential code; transient fault tolerance; Application software; Benchmark testing; Computer science; Concurrent computing; Engines; High performance computing; Libraries; Parallel processing; Parallel programming; Writing; Cluster Computing; Concurrency; Parallel Programming; Task Graph;
  • fLanguage
    English
  • Publisher
    ieee
  • Conference_Titel
    Web Society, 2009. SWS '09. 1st IEEE Symposium on
  • Conference_Location
    Lanzhou
  • Print_ISBN
    978-1-4244-4157-0
  • Electronic_ISBN
    978-1-4244-4158-7
  • Type

    conf

  • DOI
    10.1109/SWS.2009.5271793
  • Filename
    5271793