Title :
A heuristic evaluation PDS algorithm for energy-efficient delay constrained scheduling over wireless communication
Author :
Xu, Guanjun ; Jiang, Hong ; Liu, Congbin ; Wang, Yong
Author_Institution :
Inf. Inst., Southwest Univ. of Sci. & Technol., Mianyang, China
Abstract :
Hard energy constraints impose huge challenges to wireless communication under the constraint of communication delay. In this paper, we consider the problem of energy-efficient point-to-point transmission scheduling of delay sensitive multimedia data over a fading channel. We first formulate the transmission scheduling problem as a Markov Decision Process (MDP) and systematically unravel it by the optimal solution. We then propose a Heuristic Evaluation Post-decision State (HE-PDS) learning algorithm for the problem of energy-efficient scheduling with respect to the communication delay. This algorithm is based on reformulating the value iteration equation by introducing a virtual state called Post-decision State (PDS). The advantages of the proposed algorithm are that: (i) it exploits only part of the information about the system so that less information needs to be learned than when using conventional reinforcement learning algorithms; (ii) it uses the heuristic function and evaluation function to reduce unnecessary action exploration in the whole search space, which severely limits the adaptation speed and runtime performance of conventional reinforcement learning algorithms; (iii) it chooses actions with favorable purpose and efficiency so as to optimize reward function and enhance convergence speed. We compare our algorithm with the traditional Q learning algorithm and PDS learning algorithm in terms of the energy consumption and transmission delay. The simulation results illustrate the performance of the proposed algorithm under various scenarios achieves a better trade-off between energy consumption and delay overhead. Moreover, it can converge in a reasonable number of slots for it to be practically useful with satisfaction of delay constraint.
Keywords :
Markov processes; delays; fading channels; learning (artificial intelligence); multimedia communication; optimisation; radiocommunication; telecommunication computing; Markov decision process; delay constraint; delay overhead; delay sensitive multimedia data; energy efficient delay constrained scheduling; evaluation function; fading channel; hard energy constraints; heuristic evaluation postdecision state algorithm; heuristic evaluation postdecision state learning algorithm; heuristic function; point-to-point transmission scheduling; transmission scheduling problem; wireless communication; Algorithm design and analysis; Delay; Energy consumption; Energy efficiency; Heuristic algorithms; Scheduling; Wireless communication; constrained markov decision process; energy-efficient scheduling; heuristic evaluation algorithm; reinforcement learning;
Conference_Titel :
Communications (ICC), 2012 IEEE International Conference on
Conference_Location :
Ottawa, ON
Print_ISBN :
978-1-4577-2052-9
Electronic_ISBN :
1550-3607
DOI :
10.1109/ICC.2012.6364901