Balancing ILP and TLP in SMT Architectures through Out-of-Order Instruction Dispatch

Simultaneous multi-threading (SMT) architectures open up new avenues for datapath optimizations due to the presence of thread-level parallelism (TLP). One recent proposal for exploiting such parallelism is the 2OP_BLOCK scheduler design, which completely avoids the dispatch of instructions with two non-ready source operands into the issue queue. This technique reduces the scheduler complexity and also provides performance benefits for workloads with sufficient TLP, as the issue queue is more efficiently utilized. In this paper we first revisit the 2OP_BLOCK scheduler and show that this design actually results in performance losses for workloads with a limited number of threads because the constraints imposed on the exploitable ILP within each thread outweigh its advantages. To balance the ILP (instruction-level parallelism) and TLP in SMT processors supporting such schedulers, we propose out-of-order dispatch of instructions within each thread. This simple augmentation naturally allows the 2OP_BLOCK scheduler to perform well even when the number of threads is small. Furthermore, for environments with a larger number of threads, the out-of-order dispatch mechanism improves the performance of the original proposal by up to 15% on the average across simulated multithreaded mixes of SPEC 2000 benchmarks