How to run experiments with large peer-to-peer data structures

Summary form only given. Distributed hash tables (DHT) promise to administer huge sets of (key, value)-pairs under high workloads. DHT currently are a hot topic of research in various disciplines of computer science. Experimental results that are convincing require evaluations with large DHT (i.e., more than 100,000 nodes). However, many studies confine themselves to (less convincing) experimental examinations with much fewer nodes. Information on how to run experiments with DHT with many nodes is not available. Based on experience gained with a DHT implementation of our own, this article describes how to carry out such experiments successfully. The infrastructure used is a cluster of 32 commodity workstations. We start by compiling requirements regarding such experiments. We then identify the various bottlenecks that may be the result of a naive implementation, and we describe their negative effects. We propose various countermeasures, e.g., an experiment clock, and a component that maintains persistent network connections between cluster nodes. The features proposed are beneficial: A naive experimental setup allows for 10,000 peers maximum and a total of 20 operations per second, a sophisticated one following our proposal for 1,000,000 peers and 150 operations per second. Furthermore, we say why experimental results gained in such a way are meaningful in many situations.