Title :
Self-segmentation of sequences: automatic formation of hierarchies of sequential behaviors
Author :
Sun, Ron ; Sessions, Chad
Author_Institution :
Dept. of Comput. Eng. & Comput. Scil, Missouri Univ., Columbia, MO, USA
fDate :
6/1/2000 12:00:00 AM
Abstract :
The paper presents an approach for hierarchical reinforcement learning that does not rely on a priori domain-specific knowledge regarding hierarchical structures. Thus, this work deals with a more difficult problem compared with existing work, It involves learning to segment action sequences to create hierarchical structures (for example, for the purpose of dealing with partially observable Markov decision processes, with multiple limited-memory or memoryless modules). Segmentation is based on reinforcement received during task execution, with different levels of control communicating with each other through sharing reinforcement estimates obtained by each other. The algorithm segments action sequences to reduce non-Markovian temporal dependencies, and seeks out proper configurations of long- and short-range dependencies, to facilitate the learning of the overall task. Developing hierarchies also facilitates the extraction of explicit hierarchical plans. The initial experiments demonstrate the promise of the approach
Keywords :
cognitive systems; learning (artificial intelligence); Markov decision processes; action sequences; cognitive agents; reinforcement learning; sequential behaviors; Artificial intelligence; Costs; Decision making; Learning; Observability; Sun;
Journal_Title :
Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on
DOI :
10.1109/3477.846230