Transceiver Design for AF MIMO Two-Way Relay Systems with Imperfect Channel Estimation

We address the transceiver design problem for amplify-and-forward (AF) MIMO two-way relay systems with imperfect channel estimation. Since only estimated channel state information (CSI) is available, the self-interference (SI) cannot be completely canceled at destination nodes. With both channel estimation errors and residual self-interference considered, we propose two robust schemes to minimize the average sum mean squared error (MSE). % averaged over channel uncertainties., In the iterative scheme, the relay precoder and destination receivers are alternately optimized until convergence. In the constrained structure scheme, a specific structure is imposed on the relay precoder to reduce complexity. By solving a relaxed optimization problem, we derive a closed-form solution for the constrained structure precoder. Simulation results show that the proposed iterative scheme provides better robustness against channel uncertainties than the non-robust iterative scheme, and the constrained structure scheme achieves close performance with the proposed iterative scheme.