Pushing the limits of PCI-express: A PCIe application within an IBM supercomputing environment

Performance demands placed on high-speed computers raise the requirements of sub-system interfaces. While many design practices exist to maximize sub-system performance, cost and schedule limitations can reduce the variables designers have to enable targeted bus speeds. In the case of IBM´s new petaflop supercomputer, analysis revealed the strategic nature of the PCI Expressreg interface for this design. This is due to the channel variables´ ability to be modified to improve performance without impacting overall system architecture. Therefore, this paper will discuss the research and development activities of the PCIereg channel within the IBM Roadrunner petaflop supercomputer to be delivered to Los Alamos National Laboratory in 2008 (http://www.lanl.gov/roadrunner/). These development activities include novel methods aimed at reducing channel insertion loss, with minimal impact to system hardware, schedule, and cost, through utilization of flexible circuit interconnect. IBM flex circuit research and development activities will be described, as well as the channel attenuation improvements realized as a result of this research. Finally, impacts to IBM Roadrunner manufacturing, cost and qualification activities, as a result of new flex circuit constructs, will be discussed.