Weighted discounted dynamic programming

The authors consider a discrete-time Markov decision process with an infinite horizon. They maximize the sum of a number of standard discounted rewards, each with a different discount factor. It is shown that with this criterion for some positive ε there need not exist an ε-optimal stationary strategy, even when the state and action sets are finite. However, ε-strategies exist under weak conditions, ε-optimal Markov strategies are exhibited, which are stationary and some time onward. When both state and action are finite, there exists an optimal Markov strategy with this property. An explicit algorithm for the computation of such strategies is included