Applying Reinforcement Learning for Resolving Ambiguity in Service Composition

Automatically composing service-based software solutions is still a challenging task. Functional as well as non-functional properties have to be considered in order to satisfy individual user requests. Regarding non-functional properties, the composition process can be modeled as optimization problem and solved accordingly. Functional properties, in turn, can be described by means of a formal specification language. State-space based planning approaches can then be applied to solve the underlying composition problem. However, depending on the expressiveness of the applied formalism and the completeness of the functional descriptions, formally equivalent services may still differ with respect to their implemented functionality. As a consequence, the most appropriate solution for a desired functionality can hardly be determined without considering additional information. In this paper, we demonstrate how to overcome this lack of information by means of Reinforcement Learning. In order to resolve ambiguity, we expand state-space based service composition by a recommendation mechanism that supports decision-making beyond formal specifications. The recommendation mechanism adjusts its recommendation strategy based on feedback from previous composition runs. Image processing serves as case study. Experimental results show the benefit of our proposed solution.