A monitoring system for federated clouds

Cloud computing mechanisms are steadily gaining significance in the field of IT infrastructure hosting and maintenance. To date, we have reached a point where a paradigm shift can be observed. An increasing number of IT players are rethinking the way the technical foundation of their offerings is operated from in-house datacenter hosting towards the outsourcing of infrastructure or services to specialized external cloud computing companies. With the emergence of more and more commercial cloud providers and offerings, combining resources from two or more providers - in order to benefit from factors like price differences, locality of resources, etc. - becomes increasingly feasible for consumers of cloud services. Therefore, it is desirable for such users to have an overarching monitoring system that aggregates a multitude of measurements from resources of different administrative domains in a unified manner. This paper introduces a comprehensive monitoring solution for federated clouds that provides data for both infrastructure providers and cloud users. This system does not only support monitoring resources from heterogeneous domains on both the network and infrastructure level, but moreover provides monitoring support that is able to operate across large numbers of end-to-end resources at the service as well as the application level. In this paper the design of this system as well as its implementation is discussed, and a validation of the system within the context of the European funded project BonFIRE is presented. The utility of the system is shown through the conduct of experimentation.