Formation and activation of feature hierarchies under reinforcement

Representation of knowledge through a hierarchy of re-used elements, and the discovery of intermediate terms for learning, is an area of increasing interest in artificial learning. Such a hierarchy is a recognised aspect of human visual processing and has an important role in recognition of objects. A hierarchy allows efficiency of representation, and a manner of preserving links between related concepts. The use of such an approach in an artificial system requires addressing processes for discovery of features, and for activation of features according to an observation. Learning Classifier Systems provide a means of developing a population of rules relevant to a task according to reinforcement, capturing features of the problem in a population of rules. Implementation of a hierarchical representation to define rules is examined using the Activation-Reinforcement Classifier System, acting in a game environment. Two methods of activation of fragments are examined, one using a parallel activation method allowing multiple interpretations to be active in tandem, the other based on attention to a single higher level concept at once, using a limited working memory. Attention to a high level rule provides a bias on the low level features to be activated. Trials show the system operates successfully on the game of Dots and Boxes with a large game size, and is able to extract relevant features of the game using a body of 4000 autonomously produced features. The attention based activation method operates with a reduced memory requirement and faster processing time than the parallel method. The network of features produced shows a scale-free connectivity distribution, a common property of many human semantic networks.