A Zeroth-Level Classifier System for Real Time Strategy Games

Real Time Strategy games (RTS) provide an interesting test bed for agents that use Reinforcement Learning (RL) algorithms. From an agent´s point of view, RTS games constitute a Markovian, partially observable and dynamic environment with a huge state space. In this paper, we present an agent that uses a Zeroth-level Classifier System (ZCS) in order to construct winning policies for this type of games. We also combine ZCS with the replacing traces method in an attempt to improve the behaviour of our agent. We tested the learning abilities of our agent against a static opponent. For the evaluation of our agent, we compare its results with those of a random-acting agent and an agent that uses the SARSA RL algorithm. Results are encouraging since, our ZCS agent managed to outperform the SARSA agent. On the other hand, applying replacing traces to ZCS did not yield the expected results.