A framework of an agent planning with reinforcement learning for e-pet

E-pet is an animal-type robot companions, he can be physical or electronic. Reinforcement learning (RL) can be applied to the e-pet. However, the interactive instruction is constituted by complex activities. In this study, we proposed a framework that integrated AI planning technology into RL to generate the solution. In the framework, the e-pet interacts with human and includes two components: environment and agent. The agent exploits AI planning to seek goal state and Markov decision process (MDP) to choose the action and updates each Q-value using Q-learning algorithm. And we proposed the three-level subsumption architecture which including instinct level, perception level, and planning level. We build layers corresponding to each level of competence and can simply add a new layer to an existing set to move to the next higher level of overall competence. We implement the e-pet in a 3D model and train the agent. Experimental result shows that the update of Q-table reduces the number of planning states in the framework.