A Proposal for an Abstract Model Building Using Inductive Logic Programming

Various ways of abstraction in reinforcement learning methods have been proposed. The central idea is to make use of the inherent structure in the MDP itself. Most traditional techniques do not scale up to even larger domains consisting of objects and relations. We present a proposal for abstract model building to construct relational Markov decision process. This approach separates the structural induction of the representation from the actual value function estimation. First a set of first-order features is induced utilizing inductive logic programming. These are then used as input for a regression algorithm that estimates Q-value functions per action in the induced states and determine a policy. In this way we hope to improve performance of standard Q-learning.