A Markovian game-theoretical power control approach in cognitive radio networks: A multi-agent learning perspective

Due to the lack of learning and adaptation abilities in traditional game models, e.g., the strategic Nash game, they can´t describe the dynamic behaviors and strategy interactive selections well in the cognitive context. The Markov game theoretical modeling approach is investigated to deal with the power control in the cognitive radio (CR) context, which well captures the learning and adaptation abilities of CRs. With the complex interaction relationship of multiple secondary users (SUs), multiple primary users (PUs) with wireless coexistent environment into consideration, both the secondary overall utility maximization and fairness among the SUs are considered from the mathematical model formulation and the algorithm design perspective. A power control approach to searching for the fair and optimal Nash equilibrium solution (NES) based on the improved multi-agent Q-learning is proposed. Meanwhile, the parameters of the presented algorithm are analyzed through simulations. The numerical results confirm that the proposed algorithm can improve the system utility and also guarantee the fairness among the SUs well.