A POMDP framework for human-in-the-loop system

Human operators are involved in many real world systems such as automobile systems. Traditional human-assistance features such as warning systems in the aircraft and automatic braking systems in automobile only monitor the states of the machine in order to prevent human errors and enhance safety. We believe that next generation systems should be able to monitor both the human and the machine and give an appropriate feedback to them. Although having human in the control loop has its advantage, it lacks a unified modeling framework to manage the feedback between the human and the machine. In this paper, we will present how partially observable Markov decision process (POMDP) can be used as a unified framework for the three main components in a human-in-the-loop control system-the human model, the machine dynamic model and the observation model. We use simulations to show the benefits of this framework. Finally, we outline the key challenge to advance this framework.