MTCM design with nonlinear hierarchical space-time block codes

In many wireless communication systems, space-time block codes (STBC) alone are insufficient to provide the needed protection from the effects of a fading channel, and hence are more suitable as an inner code for a serially concatenated system. Serial concatenation introduces memory between STBCs indirectly. A direct approach, multiple trellis coded modulation (MTCM), can match the distance spectrum of constituent codes to the trellis structure and optimize coding gain. Full diversity is naturally maintained given the full rank of the constituent STBCs. Previous MTCM designs of STBC relied heavily on orthogonal codes. In this work, nonlinear hierarchical codes (NHC), with no constraint on the size and structure of each block code, serve as a natural candidate for set partitioning and expansion in MTCM design. Regular MTCM design procedures are proposed to exploit the layered structure of NHC which leads to optimized designs for various rates/block sizes/constellation sizes. Several factors which affect performance are analyzed and exploited to improve the overall performance. Set expansion is used to improve high rate MTCM designs. In each design, the performance/rate/complexity tradeoffs are gracefully balanced and optimized. With limited growth in complexity, the proposed designs can achieve more than 3.5 dB of gain over current MTCM designs.