Towards the Asymptotic Sum Capacity of the MIMO Cellular Two-Way Relay Channel

In this paper, we consider the transceiver and relay design for the multiple-input multiple-output (MIMO) cellular two-way relay channel (cTWRC), where a multi-antenna base station (BS) exchanges information with multiple multiantenna mobile stations via a multi-antenna relay station (RS). We propose a novel two-way relaying scheme to approach the sum capacity of the MIMO cTWRC. A key contribution of this work is a new nonlinear lattice-based precoding technique to precompensate the interstream interference, so as to achieve efficient interference-free lattice decoding at the relay. We derive sufficient conditions for the proposed scheme to asymptotically achieve the sum capacity of the MIMO cTWRC in the high signal-to-noise ratio (SNR) regime. To fully exploit the potential of the proposed scheme, we also investigate the optimal power allocation at the BS and the RS to maximize the weighted sum-rate of the MIMO cTWRC in the general SNR regime. It is shown that the problem can be formulated as a monotonic program, and a polyblock outer approximation algorithm is developed to find the globally optimal solution with guaranteed convergence. We demonstrate by numerical results that the proposed scheme significantly outperforms the existing schemes and closely approaches the sum capacity of the MIMO cTWRC in the high SNR regime.