On the optimality of an index policy for bandwidth allocation with delayed state observation and differentiated services

In this paper we study the optimality of an index policy for a bandwidth allocation problem, where a single server is allocated among N queues in a slotted system based on the queue backlog information. Due to the physical nature of the system this information is delayed, in that when the allocation decision is made, the server only has the backlog information from an earlier time. This results in imperfect and partial state observation. Queues have Bernoulli arrival processes with different probabilities of arrival, as well as different buffering/holding costs to differentiate heterogeneous classes of traffic/service. The objective is to minimize the expected total discounted holding cost over a finite or infinite horizon. We introduce an index policy with indices defined as functions of the state of a queue. We first show that when the state of the system is away from the boundary, i.e., no empty queues, the index policy is optimal. When there are empty queues, we show that under sufficient separation of the indices the index policy is still optimal. We show by example that if the separation does not hold, the index policy is not necessarily optimal. We then formulate the optimal bandwidth allocation as a restless bandit problem and show under what conditions the index policy calculated using Whittle´s heuristics, which in general is only asymptotically optimal, is optimal for the finite case