An adaptation of particle swarm optimization for Markov decision processes

In this paper, we adapt the metaheuristic of particle swarm optimization (PSO) for solving nonstochastic optimization problems into a novel convergent algorithm for solving Markov decision processes (MDP) with infinite horizon discounted cost criterion. We show that the algorithm converges to an optimal policy with probability one. We further study how to adapt PSO to develop PSO-based reinforcement learning for the case where transition and cost dynamics of a given MDP are unknown to the decision maker.