Title :
Designing multi-leader-based Allgather algorithms for multi-core clusters
Author :
Kandalla, Krishna ; Subramoni, Hari ; Santhanaraman, Gopal ; Koop, Matthew ; Panda, Dhabaleswar K.
Author_Institution :
Dept. of Comput. Sci. & Eng., Ohio State Univ., Columbus, OH, USA
Abstract :
The increasing demand for computational cycles is being met by the use of multi-core processors. Having large number of cores per node necessitates multi-core aware designs to extract the best performance. The Message Passing Interface (MPI) is the dominant parallel programming model on modern high performance computing clusters. The MPI collective operations take a significant portion of the communication time for an application. The existing optimizations for collectives exploit shared memory for intra-node communication to improve performance. However, it still would not scale well as the number of cores per node increase. In this work, we propose a novel and scalable multi-leader-based hierarchical Allgather design. This design allows better cache sharing for Non-Uniform Memory Access (NUMA) machines and makes better use of the network speed available with high performance interconnects such as InfiniBand. The new multi-leader-based scheme achieves a performance improvement of up to 58% for small messages and 70% for medium sized messages.
Keywords :
application program interfaces; cache storage; message passing; parallel programming; shared memory systems; cache sharing; computational cycles; message passing interface; multi-core clusters; multi-core processors; multi-leader-based Allgather algorithms; nonuniform memory access machines; parallel programming; Algorithm design and analysis; Bandwidth; Clustering algorithms; High performance computing; Kernel; Libraries; Message passing; Multicore processing; Parallel programming; Sockets;
Conference_Titel :
Parallel & Distributed Processing, 2009. IPDPS 2009. IEEE International Symposium on
Conference_Location :
Rome
Print_ISBN :
978-1-4244-3751-1
Electronic_ISBN :
1530-2075
DOI :
10.1109/IPDPS.2009.5160896