Capturing inter-application interference on clusters

Cluster systems usually run several applications-often from different users-concurrently, with individual applications competing for access to shared resources such as the file system or the network. Low application performance is therefore not always the result of inefficient program design, but may instead be caused by interference from outside. However, knowing the difference is essential for an appropriate response. Unfortunately, traditional performance-analysis techniques consider an application always in isolation, without the ability to compare its performance to the overall performance conditions on the system when it was executed. In this paper, we present a novel approach of how to correlate the performance behavior of applications running side by side. To accomplish this, we divide the application runtime into fine-grained time slices whose boundaries are synchronized across the entire system. Mapping performance data related to shared resources onto these time slices, we are able to establish the simultaneity of their usage across jobs, which can be indicative of inter-application interference. Our experiments show that such interference effects, for which the developer is usually not to blame, can degrade application performance significantly.