Title :
On the structure of decentralized controllers in networked MDPs
Author_Institution :
Dept. of Control & Dynamical Syst, Caltech, Pasadena, CA, USA
Abstract :
This work considers the problem of constructing optimal decentralized controllers for networked Markov Decision Processes (MDPs). A number of subsystems, each modeled as an MDP, are dynamically coupled through a network and affect one another with delay. Each subsystem has a corresponding controller which has perfect knowledge of the local state and may communicate its state to the other controllers over a noise-free channel with fixed delay. The problem is framed in terms of a coordinator with access to the shared information of the controllers, leading to a dynamic programming problem to calculate optimal policies. Further, under the criteria of partial nestedness the dynamic program exhibits additional factored structure, leading to more efficient synthesis of the optimal policy.
Keywords :
Markov processes; decentralised control; dynamic programming; networked control systems; optimal control; MDP; dynamic programming problem; fixed delay; local state; networked Markov decision processes; noise-free channel; optimal decentralized controllers; optimal policies; partial nestedness; Bayes methods; Delays; Dynamic programming; History; Markov processes; Optimal control; Random variables;
Conference_Titel :
Decision and Control (CDC), 2013 IEEE 52nd Annual Conference on
Conference_Location :
Firenze
Print_ISBN :
978-1-4673-5714-2
DOI :
10.1109/CDC.2013.6760815