Title :
TCCluster: A Cluster Architecture Utilizing the Processor Host Interface as a Network Interconnect
Author :
Litz, Heiner ; Thuermer, Maximilian ; Bruening, Ulrich
Author_Institution :
Comput. Archit. Group, Univ. of Heidelberg, Heidelberg, Germany
Abstract :
So far, large computing clusters consisting of several thousand machines have been constructed by connecting nodes together using interconnect technologies as e.g. Ethernet, Infiniband or Myrinet. We propose an entirely new architecture called Tightly Coupled Cluster (TCCluster) that instead uses the native host interface of the processors as a direct network interconnect. This approach offers higher bandwidth and much lower communication latencies than the traditional approaches by virtually integrating the network interface adapter into the processor. Our technique neither applies any modifications to the processor nor requires any additional hardware. Instead, we use commodity off the shelf AMD processors and exploit the HyperTransport host interface as a cluster interconnect. Our approach is purely software based and does not require any additional hardware nor modifications to the existing processors. In this paper, we explain the addressing of nodes in such a cluster, the routing within such a system and the programming model that can be applied. We present a detailed description of the tasks that need to be addressed and provide a proof of concept implementation. For the evaluation of our technique a two node TCCluster prototype is presented. Therefore, the BIOS firmware, a custom Linux kernel and a small message library has been developed. We present microbenchmarks that show a sustained bandwidth of up to 2500 MB/s for messages as small as 64 Byte and a communication latency of 227 ns between two nodes outperforming other high performance networks by an order of magnitude.
Keywords :
Linux; firmware; multiprocessor interconnection networks; network interfaces; software libraries; telecommunication network routing; workstation clusters; BIOS firmware; Ethernet; HyperTransport host interface; Infiniband; Linux kernel; Myrinet; TCCluster; cluster architecture; cluster interconnect; communication latency; computing cluster; interconnect technology; message library; native host interface; network interconnect; network interface adapter; processor host interface; programming model; shelf AMD processor; system routing; tightly coupled cluster; Bandwidth; Hardware; Message passing; Programming; Protocols; Routing; Scalability; AMD; HPC; HyperTransport; Low latency; Opteron; high bandwidth; interconnect;
Conference_Titel :
Cluster Computing (CLUSTER), 2010 IEEE International Conference on
Conference_Location :
Heraklion, Crete
Print_ISBN :
978-1-4244-8373-0
Electronic_ISBN :
978-0-7695-4220-1
DOI :
10.1109/CLUSTER.2010.37