Title :
HAM homomorphism for state abstraction
Author :
Du Xiaoqin ; Qinghua, Li ; Jianjun, Han
Author_Institution :
Wuhan Univ. of Sci., Wuhan Univ. of Sci. & Eng., Wuhan, China
Abstract :
In the HRL field, there are several main methods such as HAMs, options, MAXQ. A main problem that exists in HAMs is its joint state space consisting of the cross-product of the machine states in the HAM and the states in the original MDP, which can not be completely solved by a subroutine-based state abstraction method. This paper analyzes this problem in detail, provides formal definitions of homomorphism in HAMs and proves the invariance of the optimal solution for HAMs. Several typical examples are analyzed and evaluated. The results show that HAM homomorphism can conquer this problem.
Keywords :
finite automata; learning (artificial intelligence); HAM homomorphism; hierarchical abstract machine; hierarchical reinforcement learning; state abstraction; Accelerated aging; Algorithms; Automation; Computer science; Educational institutions; Learning; Logistics; State-space methods; HAMs; Hierarchical Reinforcement Learning; Homomorphism;
Conference_Titel :
Automation and Logistics, 2009. ICAL '09. IEEE International Conference on
Conference_Location :
Shenyang
Print_ISBN :
978-1-4244-4794-7
Electronic_ISBN :
978-1-4244-4795-4
DOI :
10.1109/ICAL.2009.5262638
