Channel probing in communication systems: Myopic policies are not always optimal

We consider a multi-channel communication system in which a transmitter has access to a large number of channels, but does not know the state of these channels. We model channel state using an ON/OFF Markovian model, and allow the transmitter to probe one of the channels at predetermined probing intervals to decide over which channel to transmit. For models in which the transmitter must send over the probed channel, it has been shown that a myopic policy that probes the channel most likely to be ON is optimal. In this work, we allow the transmitter to select a channel over which to transmit that is not necessarily the one it probed. We show that the myopic policy is not optimal, and propose a simple alternative probing policy, which achieves a higher per-slot expected throughput. Finally, we consider the case where there is a fixed cost associated with probing and derive optimal probing intervals.