Is XCS Suitable For Problems with Temporal Rewards?

XCS, the accuracy-based classifier system, provides a very brilliant way to merge genetic algorithmic (GA) rule learning and reinforcement learning (RL) methodologies together. This makes it suitable for a wide range of applications where generalisation over decision making states is desirable. Also, its Q-learning-oriented prediction update scheme enables it to handle multi-step problems adequately. This paper reports how the intertwined spirals problem, initially a popular benchmark in classification, was modified by the authors to verify XCS´s suitability for behavioural design of robotic systems. When the results obtained were not as expected, investigations were continued until a rather surprising conclusion was drawn: XCS cannot handle very simple problems if the rewards are temporally-oriented, even if the reward is extremely short-delayed