Performance variations of two open-source cloud platforms

The performance of workloads running on cloud platforms varies significantly depending on the cloud platform configurations. We evaluated the performance variations using two open-source cloud platforms, OpenNebula and Eucalyptus. To assess the performance variations on the cloud platforms, we created a representative workload from Wikipedia software and data. The performance with this workload was quite sensitive to two key configuration choices, the physical location of the virtual machine disk images (local disk or NFS), and eager or lazy allocation of the virtual machine disk images. Our performance metrics included (1) the provisioning times for the virtual machines, (2) the elapsed times for two types of batch processing, and (3) the throughputs of two types of Web transactions. The local-disk configuration was 75% slower for provisioning, 2.9 times faster for batch possessing, and 50% faster for Web transactions compared to the NFS configuration. Relative to lazy-allocation, eager-allocation took 2.7 times longer for provisioning and was 43% faster for batch processing, but was only 1.5% faster for Web transactions. Our results indicate that no configuration offers the best performance for all three of the metrics at the same time. If batch processing is more important than provisioning, the local-disk configuration with eager disk allocation should be used. Otherwise, local-disk allocation with lazy allocation should be used. We also evaluated a multi-tenancy scenario using the Apache Day Trader benchmark on Eucalyptus. The results show that VM provisioning significantly affected the throughputs of Day Trader due to the lack of any disk I/O throttling mechanism.