Improving resource efficiency in data centers using reputation-based resource selection

Nowadays, data centers are consuming a lot of energy but not in an efficient fashion. Much of energy is wasted. There are several types of energy waste at different levels including infrastructure-, machine- and system-level waste. The former two levels have been improved significantly in the last few years, however, few efforts have been put on the last level, especially the resource waste caused by failures in a data center. In this paper, we attack the problem proactively by leveraging a reputation-based resource selection scheme to reduce the number of resubmissions of tasks, resulting from the failure during the course of their execution. To capture the characteristics of resources, we introduce Opera, an OPEn ReputAtion model. Opera characterizes itself with two important novelties. One is that Opera employs a vector, instead of a single value, to represent the reputation of a resource (entity) in order to capture its heterogeneity in different points of view. The other novelty of Opera is introducing the just-in-time feature that represents the real-time system status, which, to our knowledge, has never been considered in conventional reputation systems. To demonstrate the effectiveness of Opera, we have integrated and implemented the Opera trust model with the scheduler in Hadoop, a popular, data intensive open source framework. The experimental results showed that Opera enabled the scheduler to select appropriate nodes to assign tasks based on different criteria, and this helped reduce not only the number of re-executed tasks but also the execution time of Hadoop´s jobs under the presence of failures and heavy workload up to 59% and 32%, respectively.