Adding memory to XCS

We add internal memory to XCS (eXtended Classifier System). We then test this version of XCS with internal memory, named XCSM, in non-Markovian environments with two and four aliasing states. The experimental results show that XCSM can easily converge to optimal solutions in simple environments; moreover, XCSM´s performance is very stable with respect to the size of the internal memory involved in learning. However, the results we present evidence that in more complex non-Markovian environments, XCSM may fail to evolve an optimal solution. Our results suggest that this happens because the exploration strategies currently employed with XCS are not adequate to guarantee the convergence to an optimal policy with XCSM in complex non-Markovian environments