Parallel programming models for finite-element method using preconditioned iterative solvers with multicolor ordering on various types of SMP cluster supercomputers

Parallel iterative linear solvers for unstructured grids in FEM application, which were developed for the Earth Simulator (ES), have been ported to various types of SMP cluster supercomputers. Performance of flat-MPI and hybrid parallel programming model has been compared using more than 100 SMP nodes of ES, Hitachi SR8000 and IBM SP-3. Effect of coloring and method for storage of coefficient matrices have been also evaluated in various types of applications. Generally speaking, codes, which are developed for vector processors using multicolor ordering, demonstrate better performance with many colors on scalar processors. Flat-MPI and hybrid are competitive for each supercomputer. On ES, hybrid outperforms flat-MPI when number of SMP node is large and problem size is small because of MPI latency. In parallel computation of FEM-type applications, MPI latency is a very important parameter for choosing flat-MPI or hybrid