Minimum delay routing in stochastic networks

The authors consider the problem of traveling with least expected delay in networks whose link delays change probabilistically according to Markov chains. This is a typical routing problem in dynamic computer communication networks. They formulate several optimization problems, posed on infinite and finite horizons, and consider them with and without using memory in the decision making process. It is proved that all these problems are, in general, intractable. However, for networks with nodal stochastic delays, a simple polynomial optimal solution is presented. This is typical of high-speed networks, in which the dominant delays are incurred by the nodes. For more general networks, a tractable ε-optimal solution is presented. The performance of a regular shortest-path algorithm in such an environment is considered