Using Intelligent Multi-Agent Systems to Model and Foster Self-Regulated Learning: A Theoretically-Based Approach Using Markov Decision Process

In self-regulated learning concept, Intelligent Tutoring Systems (ITS) can be designed to foster learning behaviors through pedagogical agents (PAs) that are used for interactions and exchange information with the human learner. These agents are intelligent and follow rational behaviors, but in the case of multi-agent environments they need to be systematically and specifically designed, however in order to follow a common goal, different self-regulatory systems have been designed that use pedagogical agents, but they fail to constrain the decision making of the agents and maintain a sequential decision making process during learning interactions with human learners. In this paper, we provide a new theoretical model for agent-learner interactions in MetaTutor, a multi-agent hypermedia learning environment, using Markovdecision processes. We theoretically define the agents´ Markovdecisions and their influence on MetaTutor´s performance as a whole. First, we formally define the Markov architecture and its parameters. We then link these characteristics to the pedagogical agents we use in MetaTutor and define different versions of MetaTutor agents equipped with Markov decision mechanism. Furthermore, we explore additional details about agents´ sequential decision making and how reward functions influence their acting strategies with learners in the learning environment. We introduce the optimization problem in which we aim to maximize the expected return of the overall agents´ acts in a self-regulatory system. What specifically distinguishes this work from the previous proposals in the same domain is its novelty in continuous decision making mechanism investigation and performance analysis that improve the applicability of the proposed adaptive model in a multi-agent ITS like MetaTutor.