Protocol-independent event building evaluator for the LHCb DAQ system

The Data Acquisition (DAQ) system of LHCb is a complex, real-time system connecting more than 300 readout boards with more than 1500 servers by means of high-performance network 68 switches. The total data rate through this network is more than 60 GB/s. The DAQ network has a FAT-tree topology, composed of an aggregation and a distribution layer, where traffic flows unidirectionally to the server farm. This system will be upgraded to provide LHCb with an all-software, trigger-free readout starting from 2020. Consequently more CPU power in the from of servers are needed and the DAQ network will grow to a capacity of 40 Tbps. A PC-based readout system would receive data incoming from the detector, which would then be scattered across builder nodes, and further distributed to a computing farm for data filtering. The design bandwidth of such a DAQ system requires rates as high as 400 Gbps single-duplex per node. These builder nodes will be connected with inexpensive, high-bandwidth data-centre switches in order to minimize the system cost. The behaviour of such a event-building network can of course be studied in simulation but experience tells us that it is imperative to test, in particular to find out limitiations in the switches themselves and to which extent various event-building protocols can mitigate these limitations. We present a protocol, topology and transport independent emulation software DAQPIPE. It allows us to test different communication architectures, such as push or pull, with regards to the initiator of the communication. Different topologies and transport protocols can also be tested. We also present throughput and stress tests on an Infiniband FDR multi-rail based LAN network setup, with a focus on the network performance.