Introduction

Gromacs is available on the Maestro cluster via a module. It also needs the modules gcc, cuda, and openmpi to be loaded. We regularly update the installed version. If you plan on using Gromacs extensively, please get in touch with us to sort out the details.

Running

The most common situation for running Gromacs is on 16 CPU cores and one GPU. You should always start in this configuration to establish a baseline for the performance.  Here is an example sbatch file for gromacs:

gromacs.sbatch (bash)

#!/bin/bash
### name of the job
#SBATCH --job-name=100yearsjob
### number of nodes
#SBATCH --nodes=1
### Specify the number of tasks per node. Defines the number of MPI ranks per node
#SBATCH --ntasks-per-node=1
### Specify the number of cpus per task. Defines the number of OpenMP threads per MPI rank
#SBATCH --cpus-per-task=16
#SBATCH --hint=nomultithread
### type of partition/qos
####SBATCH --account=<youraccounthere> 
#### only if you need to specify non-default account
#SBATCH --partition=gpu
#SBATCH --qos=gpu
# request 1 GPU
#SBATCH --gres=gpu:1

# setting up modules
module load gcc/9.2.0
module load openmpi/4.0.5
module load cuda/12.2
module load gromacs/2024.2

# export some variables
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

srun --cpu-bind=socket gmx mdrun --deffnm npt_posres -nsteps 5000000 -ntomp $OMP_NUM_THREADS -update gpu

Note that, compared to most tutorials available online, we use srun instead of mpirun and gmx instead of gmx_mpi. Srun handles MPI mechanics, and the unified executable gmx can be used with or without MPI. Do not use more than 16 cores per GPU, as it would render other GPUs on the same server inaccessible.

If you prefer running Gromacs on the CPU only, we suggest starting with 48 threads and 1 MPI task, then experimenting with 2 and 4.

gromacs.sbatch (bash)

#!/bin/bash
### name of the job
#SBATCH --job-name=100yearsjob
### number of nodes
#SBATCH --nodes=1
### Specify the number of tasks per node. Defines the number of MPI ranks per node
#SBATCH --ntasks-per-node=1
### Specify the number of cpus per task. Defines the number of OpenMP threads per MPI rank
#SBATCH --cpus-per-task=48
### type of partition/qos
####SBATCH --account=<youraccounthere> 
#### only if you need to specify non-default account
#SBATCH --partition=common
#SBATCH --qos=normal
# setting up modules
module load gcc/9.2.0
module load openmpi/4.0.5
module load cuda/12.2
module load gromacs/2024.2

# export some variables
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

srun --cpu-bind=socket gmx mdrun --deffnm npt_posres -nsteps 5000000 -ntomp $OMP_NUM_THREADS

Below are optimization guides.

250250

You can also check this method here: https://pmc.ncbi.nlm.nih.gov/articles/PMC7660746/