Adaptive control of a class of nonidentifiable Markov chains

Adaptive control of finite unknown Markov chains is considered. A new performance criterion from the theory of bandit processes has recently been introduced for adaptive control of Markov chains. The new performance criterion is stronger than the expected average cost criterion and is more appropriate when the identifiability condition does not hold. An adaptive controller is derived to achieve optimality for a modified version of the new performance criterion