Parallelization of a Vector-Optimized 3-D Flow Solver for Multi-core Node Clusters

In this paper, we describe the development of a parallel version of the IR3D (Incompressible Realistic 3-D) code, which simulates the environmental effects on the evolution of vortices trailing behind control surfaces of underwater vehicles. The objective of the project was to parallelize and optimize the existing implementation for clusters of multi-core nodes. The primary motivation was to reduce turnaround time and add the capability to handle large problem sizes. Furthermore we were aiming for portability and scalability. The code solves the 3D Boussinesq equations for incompressible fluids. Fast-Fourier transforms (FFTs) are used for the calculation of horizontal derivatives and a higher-order compact finite difference scheme is used for vertical derivatives. To ensure incompressibility, the code employs a projection method for which we developed a new Poisson Solver. This solver works by computing 2D FFTs in horizontal-planes, numerically solving the resulting ordinary differential equations (ODEs) for Fourier coefficients, and then doing Fourier inversion. Parallelization is based on the Message Passing Interface (MPI) programming paradigm. We present performance and scalability results of PIR3D (Parallel IR3D) on a variety of hardware platforms and discuss methods for further optimization by exploiting additional by exploiting additional levels of parallelism.