Approximate value iteration and temporal-difference learning

In principle, a wide variety of sequential decision problems-ranging from dynamic resource allocation in communication networks to inter-temporal investment-can be formulated in terms of stochastic control and solved by the algorithms of dynamic programming. Unfortunately, due to the curse of dimensionality, the associated computational requirements become intractable in most practical contexts. The paper discusses approximate valve iteration, an algorithm that tries to alleviate the curse of dimensionality. We present potential problems associated to it and special cases in which desirable results are obtained. We also discuss the relationship between approximate value iteration and temporal-difference learning