Safety control of partially observed MDPs with applications to machine maintenance problems

In this paper we introduce the concept of safety control of partially observable Markovian systems and apply it to a class of machine maintenance problems. Given a set of constraint, an information state of a partially observed system is safe if it is in the set. A policy is safe if it makes a safe initial information state remain safe with probability 1 at each time step. The objective of safety control is to search for a safe policy and its corresponding set of safe initial information states, which is naturally a subset of the set of constraint. We first obtain a linear programming formulation to characterize the safe policy whose maximum set of safe initial information states is the entire set of constraint. Then we study the well-known machine maintenance problem under the safety control and give a numerical example to analyze the performance. Our analysis reveals some insight into the system´s behavior, which cannot be observed under the traditional optimal control