Optimal power allocation over multiple identical Gilbert-Elliott channels

In a communication system with time varying channel qualities, it is ideal to allocate the limited transmission power to channels that will be in good state. However, it is very challenging to do so because channel states are usually unknown when the power allocation decision is made. In this paper, we derive an optimal power allocation policy that can maximize the expected discounted number of bits transmitted over an infinite time span. Specifically, we first model this problem as a partially observable Markov decision processes (POMDP), and analytically investigate the structure of the optimal policy. Then a simple threshold-based policy is derived for a three-channel communication system. By formulating and solving a linear programming formulation of this power allocation problem, we further verified the derived structure of the optimal policy.