A High Performance, QoS-Enabled, S3-Based Object Store

A scale-out and reliable object store is an important building block of a cloud service, for storing virtual machine images, backups and large application data. As such an object store, Amazon S3 is available for Amazon EC2 users and other S3-compatible storage systems are also used in private clouds. However, there is concern about performance instability when many applications concurrently access the storage service, due to the characteristics of shared use. This paper presents an approach to introducing a QoS-enabled function into the S3-based object store. The object store accepts an explicit performance request as an advanced reservation, and enables QoS in the access with the extended S3 Restful interface. Implicit and static performance setting is also possible for the unmodified S3 interface. Papio S3, an object store which supports both of these S3 interfaces, is developed for implementing the approach, along with achievement of high performance upload/download using the multipart data transfer. The evaluation confirms the performance of Papio S3, and its QoS capability at the S3 data transfer, in several situations where multiple S3 clients concurrently access the same Papio S3 system. In addition, the QoS effect is compared with a load balancing approach in an existing object store, as part of the experiment.