Genetic Network Programming with updating rule accumulation

Conventional evolutionary computation methods aim to find elite individuals as the optimal solutions. The rule accumulation method tries to find good experiences from individuals throughout the generations and store them as decision rules, which is regarded as solutions. Genetic Network Programming (GNP) is competent for dynamic environments because of its directed graph structure, reusability of nodes and partially observable processes. A GNP based rule accumulation method has been studied and applied to the stock trading problem. However, with the changing of dynamic environments, the old rules in the rule pool are incompetent for guiding agent´s actions, thus updating these rules becomes necessary. This paper proposes a new method to update the accumulated rules in accordance with the environment changes. Sarsa-learning which is a good on-line learning policy is combined with off-line evolution to generate better individuals and update the rules in the rule pool. Tile world problem which is an excellent benchmark for multi-agent systems is used as the simulation environment. Simulation results demonstrate the efficiency and effectiveness of the proposed method in dealing with the changing environments.