Incorporating expert knowledge in Q-learning by means of fuzzy rules

Incorporating expert knowledge in reinforcement learning is an important issue, especially when a large state space is concerned. In this paper, we present a novel method for accelerating the setting of Q-values in the well-known Q-learning algorithm. Fuzzy rules indicating the state values will be used, and the knowledge will be transformed to the Q-table or Q-function in some first training experiences. There have already been methods to initialize the Q-values using fuzzy rules, but the rules were the kind of state-action rules and needed the expert to know about environment transitions on actions. In the method introduced in this paper, the expert should only apply some rules to estimate the state value while no appreciations about state transitions are required. The introduced method has been examined in a multiagent system which has the shepherding scenario. The obtaining results show that Q-learning requires much less iterations for getting good results if using the fuzzy rules estimating the state value.