Flat MPI vs. Hybrid: Evaluation of Parallel Programming Models for Preconditioned Iterative Solvers on “T2K Open Supercomputer”

In this work, parallel preconditioning methods based on ¿hierarchical interface decomposition (HID)¿ and hybrid parallel programming models were applied to finite-element based simulations of linear elasticity problems in media with heterogeneous material properties. Reverse Cuthill-McKee reordering with cyclic multicoloring (CM-RCM) was applied for parallelism through OpenMP. The developed code has been tested on the ¿T2K open supercomputer (Todai combined cluster)¿ using up to 512 cores. Performance of Hybrid 4 x 4 parallel programming model is competitive with that of flat MPI using appropriate command lines for NUMA control. Furthermore, reordering of the mesh data for contiguous access to memory with first touch data placement provides excellent improvement on performance of Hybrid 8 x 2 and 16 x 1, especially if the problem size for each core is relatively small. Thus, hybrid parallel programming model could be a reasonable choice for large-scale computing of sparse linear solvers on multi-core/multi-socket architectures, such as ¿T2K open supercomputer¿.