Joint Source and Relay Optimization for Two-Way Linear Non-Regenerative MIMO Relay Communications

In this paper, we investigate the challenging problem of joint source and relay optimization for two-way linear non-regenerative multiple-input multiple-output (MIMO) relay communication systems. We derive the optimal structure of the source and relay precoding matrices when linear minimal mean-squared error (MMSE) receivers are used at both destinations in the relay system. We show that for a broad class of frequently used objective functions for MIMO communications such as the MMSE, the maximal mutual information (MMI), and the minimax MSE, the optimal relay and source matrices have a general beamforming structure. This result includes existing works as special cases. Based on this optimal structure, a new iterative algorithm is developed to jointly optimize the relay and source matrices. We also propose a novel suboptimal relay precoding matrix design which significantly reduces the computational complexity of the optimal design with only a marginal performance degradation. Interestingly, we show that this suboptimal relay matrix is indeed optimal for some special cases. The performance of the proposed algorithms are demonstrated by numerical simulations. It is shown that the novel minimax MSE-based two-way relay system has a better bit-error-rate (BER) performance compared with existing two-way relay systems using the MMSE and the MMI criteria.