Optimal LLP Supervisor for Discrete Event Systems Based on Reinforcement Learning

Supervisory control is a control framework for discrete event systems. A controller, called a supervisor, restricts the behavior of the system so as to satisfy a control specification. This paper proposes an optimal LLP (limited look ahead policy) supervisory control method for discrete event systems based on reinforcement learning. A global system is composed of several local systems, and it has a logical control specification and quantitative costs for the occurrence and disabling of events. In each local system, the supervisor learns the evaluation for control patterns from the rewards. Then the supervisor makes the tree of the global system behavior based on limited look ahead and assigns the optimal control pattern based on the value function within the given specification. We show an learning algorithm of the optimal LLP supervisor and illustrate the efficiency of the proposed method by computer simulation.