Finite difference methods and Markov chain approximation to a class of robust control problems

This paper is concerned with finite difference methods and Markov chain approximation to a class of robust controlled piecewise deterministic Markov processes. The motivation comes from the investigation of production planning and controls of various manufacturing systems with unreliable machines and unknown demand. In addition to controlling the production rate, one wishes to optimize the maintenance effort under worst case demand influence. The unmodelled disturbance can be either deterministic or stochastic. We develop numerical methods for the optimal control and the value function via Markov chain approximation techniques. We show that the sequence of approximating Markov chain converges, and the sequence of approximating value functions converges to the true value function. We also provide numerical examples of manufacturing systems