Learning partially observable Markov decision model with EM algorithm

Most of existing researches focus on POMDP modeling or solution. But in some study fields, before obtaining optimal policy from a POMDP, we need first learning a POMDP model from history data. Assumed that history data including observation sequence and action sequence, the state sequence are unobservable, we derive necessary formulas for using EM Algorithm to estimate the parameters of a POMDP model, including the initial state distribution, stochastic transition matrix and observation probability function.