A Markov Decision Model for Adaptive Scheduling of Stored Scalable Videos

We propose two scheduling algorithms that seek to optimize the quality of scalably coded videos that have been stored at a video server before transmission. The first scheduling algorithm is derived from a Markov decision process (MDP) formulation developed here. We model the dynamics of the channel as a Markov chain and reduce the problem of dynamic video scheduling to a tractable Markov decision problem over a finite-state space. Based on the MDP formulation, a near-optimal scheduling policy is computed that minimizes the mean square error. Using insights taken from the development of the optimal MDP-based scheduling policy, the second proposed scheduling algorithm is an online scheduling method that only requires easily measurable knowledge of the channel dynamics, and is thus viable in practice. Simulation results show that the performance of both scheduling algorithms is close to a performance upper bound also derived in this paper.