The document discusses using Singularity containers to simplify running Delft3D FM on high performance computing clusters. Singularity containers package application code, libraries, and dependencies into a single file, avoiding the need for source code compilation. Using containers provides benefits like isolation, portability, and reproducibility. The document explains how to obtain a Delft3D FM container from Deltares, adapt scripts to interface it with the HPC scheduler and MPI library, and run simulations on the cluster. Some disadvantages are an added dependency and potentially complex configuration process.

1. Adri Mourits
Yvonne Olij-Kenyon
Thanks to Mohamed Nabi
15 November 2022
Singularity containers:
simplifying the use of
Delft3D FM on High
Performance Computing
(HPC) clusters

2. Abstract
2
The Deltares Delft3D FM kernels are currently available, precompiled, in a Singularity container. This
presentation shows briefly what a container is, why it simplifies the use of Delft3D FM, and how to use
it. The focus is on usage on High Performance Computing clusters. Disadvantages are discussed.

3. Table of contents
3
About Singularity containers:
• What is it?
• Why using it?
• How to use it for Delft3D FM?
• Disadvantages
• Future work
• Summary

4. What is it?
4
A Singularity container is an executable unit of software in which a ‘definition’ file
is used to package application code, along with libraries and dependencies.
The result is a single Singularity image file.
A single
.sif file
definition file
delft3dfm
RPM
libraries &
dependencies
lightweight
operating
system

5. What is it? Virtual Machine versus Container
5
Container:
• Less overhead
• Less resources needed

6. What is it? Different types of containers
Singularity Docker
Delft3D FM YES YES
Delft3D 4 NO YES
OS Linux Linux, Windows
Root access needed NO YES
File size 300 MB 2 GB
6
September 2022: Singularity is moved into the Linux Foundation and renamed to Apptainer

7. Why using it?
• No source code compilation needed
7

8. Why using it?
• No source code compilation needed
8
Isolation: no interference with other applications

9. Why using it?
• No source code compilation needed
9
Isolation
Lightweight: share the host kernel

10. Why using it?
• No source code compilation needed
10
Isolation
Lightweight
Flexible: Every application can be
containerized

11. Why using it?
• No source code compilation needed
11
Isolation
Lightweight
Flexible
Interchangeable: deploy updates/upgrades
on-the-fly

12. Why using it?
• No source code compilation needed
12
Isolation
Lightweight
Flexible
Interchangeable
Portable: build locally, deploy to the cloud, run anywhere

13. Why using it?
• No source code compilation needed
13
Isolation
Lightweight
Flexible
Interchangeable
Portable
Scalable: increase, automatic distribution

14. Why using it?
• No source code compilation needed
14
Isolation
Lightweight
Flexible
Interchangeable
Portable
Scalable
Stackable: vertical service stacking on-the-fly

15. Why using it?
• No source code compilation needed
15
Isolation
Lightweight
Flexible
Interchangeable
Portable
Scalable
Stackable
Reproducibility: same results, independent of
underlying hardware

16. Why using it?
• No source code compilation needed
16
Isolation
Lightweight
Flexible
Interchangeable
Portable
Scalable
Stackable
Reproducibility
Performance: close to bare-metal (without virtualization/containers)

17. How to use it for Delft3D FM? (1/5)
For Delft3D FM on HPC cluster:
1. Prepare your cluster:
1. Usually your System Admin needs to do this
2. Install Apptainer (https://apptainer.org/docs/admin/main/installation.html)
3. Install “IntelMPI Library for Linux”
It must be outside the container, on the hosting system, for optimal performance
17
The set-up, tuning and
optimising of IntelMPI is
platform dependent

18. How to use it for Delft3D FM? (2/5)
For Delft3D FM on HPC cluster:
2. From Deltares:
1. Request to software@deltares.nl
2. Receive
18
Please, give me the latest Singularity container
Readme.txt sif container
execute_singularity.sh
Keep in one folder
e.g. /opt/delft3dfm
run_singularity.sh
submit_singularity.sh
Copy to working folder

19. How to use it for Delft3D FM? (3/5)
For Delft3D FM on HPC cluster:
3. Adapt script “execute_singularity.sh”:
1. Usually your System Admin needs to do this
2. IntelMPI settings (FI_PROVIDER, I_MPI_FABRICS)
3. Queueing dependencies (Slurm, SGE)
4. Feed everything correctly to the Singularity container
• Examples are available (Deltares h6-cluster, Surf Snellius)
• IntelMPI benchmark might help
• This process might be painful, but when it’s done, updates should be straight forward!
19
! This is needed to get the expected performance

20. How to use it for Delft3D FM? (4/5)
For Delft3D FM on HPC cluster:
4. Adapt run script in your working folder:
1. “run_singularity.sh” : example without queueing system
2. “submit_singularity.sh” : example with SGE queueing system
3. Correct path to Singularity container and “execute_singularity.sh” script
4. Choose number of nodes and partitions
5. Adapt input file with Bash commands Optionally
6. Partitioning by calling “execute_singularity.sh” Optionally
7. Start computation by calling “execute_singularity.sh”
8. Merge output files by calling “execute_singularity.sh” Optionally
20
Your input
The script does

21. How to use it for Delft3D FM? (5/5)
For Delft3D FM on HPC cluster:
5. Run
By executing your run script
21
$ sbatch submit_singularity.sh
Example command for Slurm:

22. Disadvantages
22
• Another dependency
• Configuration might be painful
• Why did they have to change the name from Singularity to Apptainer!

23. Future work
23
• Simplify/generalize scripts
• Switch to Apptainer
• Use a container to compile

24. Summary
24
If you use a HPC cluster for Delft3D FM computations
Then start using Apptainer
?

25. Contact
www.deltares.nl
info@deltares.nl
@deltares
@deltares
linkedin.com/company/deltares
facebook.com/deltaresNL