Delay-optimal multi-flow buffered decode-and-forward relay communications with limited renewable energy storage

In this paper, we consider the delay minimization for multi-flow buffered decode-and-forward relay communications with renewable energy source, where the transmit power of the source nodes and the relay node is contributed by both the conventional AC utility power and the renewable power. We formulate the delay-optimal resource control problem as an infinite horizon average cost Constrained Markov Decision Process. By exploring the special problem structure, we shall derive an equivalent Bellman equation based on a reduced state space. Using a fluid approximation approach, we derive the power, rate and link selection policy which is asymptotically optimal for small slot durations. We further propose a distributed online learning algorithm to estimate the per-flow value functions as well as the Lagrange multipliers. We establish the technical proof for the almost-sure convergence of the proposed learning algorithm. By simulation, we show that the proposed scheme can achieve substantial delay performance gain compared with various conventional baseline protocols.