Convex stochastic control problems

The solution of the infinite horizon stochastic control problem under certain criteria, the functional characterization and computation of optimal values and policies, is related to two dynamic programming-like functional equations: the discounted cost optimality equation (DCOE) and the average cost optimality equation (ACOE). The authors consider what useful properties, shared by large and important problem classes, can be used to show that an ACOE holds, and how these properties can be exploited to aid in the development of tractable algorithmic solutions. They address this issue by concentrating on structured solutions to stochastic control models. By a structured solution is meant a model for which value functions and/or optimal policies have some special dependence on the (initial) state. The focus is on convexity properties of the value function