Markov-achievable payoffs for finite-horizon decision models

Consider the class of n-stage decision models with state space S, action space A, and payoff function g : (S × A)n × S → R. The function g is Markov-achievable if for any possible set of available randomized actions and all transition laws, each plan has a corresponding Markov plan whose value is at least as good. A condition on g, called the “non-forking linear sections property”, is necessary and sufficient for g to be Markov achievable. If g satisfies the slightly stronger “general linear sections property”, then g can be written as a sum of products of certain simple neighboring-stage payoffs.