MPI Launchers

Jobs that scale massively parallel and split into independent tasks rely on MPI modules and launchers to orchestrate.

The most common MPI module is Intel MPI, available through oneapi modules. Besides oneapi, OpenMPI, MPICH and others exist.

Usage of each of the module/launchers are split into each supercomputer, then by module.

Sol MPI

Intel OneAPI

module load intel/intel-oneapi-2022.1.0

mpiexec.hydra -genvall -np $SLURM_NTASKS "./my-simulation"

Intel Parallel Studio

module load intel/parallel-studio-2020.4

mpiexec.hydra -genvall -np $SLURM_NTASKS "./my-simulation"

Example script available at: /packages/public/sol-sbatch-templates/template/example-mpi-with-intel-job/main.sh

OpenMPI

module load openmpi/4.1.5

srun --export=ALL --mpi=pmix "./my-simulation"

Example script available at: /packages/public/sol-sbatch-templates/template/example-mpi-with-hpcx-job/main.sh

MPICH

module load mpich/4.2.2

mpirun -launcher slurm "./my-simulation"

Example script available at: /packages/public/sol-sbatch-templates/template/example-mpi-with-mpich-job/main.sh

MVAPICH

module load mvapich/3.0b

mpirun -launcher slurm "./my-simulation"

Example script available at: /packages/public/sol-sbatch-templates/template/example-mpi-with-mvapich-job/main.sh

Phoenix MPI

Intel OneAPI

module load intel/oneapi

mpiexec.hydra -genvall -np $SLURM_NTASKS "./my-simulation"

Example script available at: /packages/public/phx-sbatch-templates/templates/example-mpi-with-intel-job/main.sh

info

Note: There have been seen a few cases where users have needed to add the following to their job scripts: export HYDRA_LAUNCHER_EXTRA_ARGS="--export=ALL" export FI_PROVIDER=verbs #Multi-node jobs only export FI_PROVIDER=shm #Single-node jobs only

OpenMPI 4

module load openmpi/4.1.5

srun --export=ALL --mpi=pmix "./my-simulation"

Example script available at: /packages/public/sol-sbatch-templates/template/example-mpi-with-hpcx-job/main.sh

OpenMPI 5

module load openmpi/5.0.6

mpirun -np $SLURM_NTASKS "./my-simulation"

Example script available at: /packages/public/phx-sbatch-templates/templates/example-mpi-with-openmpi5-job/main.sh

MPICH

module load mpich/4.1.2

mpirun -launcher slurm "./my-simulation"

Example script available at: /packages/public/phx-sbatch-templates/templates/example-mpi-with-mpich-job/main.sh