On self-adaptive resource allocation through reinforcement learning

Autonomic computing was proposed as a promising solution to overcome the complexity of modern systems, which is causing management operations to become increasingly difficult for human beings. This work proposes the Adaptation Manager, a comprehensive framework to implement autonomic managers capable of pursuing some of the objectives of autonomic computing (i.e., self-optimization and self-healing). The Adaptation Manager features an active performance monitoring infrastructure and two dynamic knobs to tune the scheduling decisions of an operating system and the working frequency of cores. The Adaptation Manager exploits artificial intelligence and reinforcement learning to close the Monitor-Plan-Analyze-Execute with Knowledge adaptation loop at the very base of every autonomic manager. We evaluate the Adaptation Manager, and especially the adaptation policies it learns by means of reinforcement learning, using a set of representative applications for multicore processors and show the effectiveness of our prototype on commodity computing systems.