In this deck from the Linaro Connect conference, Brent Gorda presents an update on ARM for HPC. "Arm-based systems are showing up in the HPC community and new silicon is coming. The architecture has also been selected for several of the exascale projects worldwide. Brent will talk about the aspects of Arm that are attractive to the HPC community, updates on projects and what we as a community can do to help accelerate adoption in this space." Watch the video: https://insidehpc.com/2019/09/an-update-on-arm-in-hpc/ Learn more: https://developer.arm.com/tools-and-software/server-and-hpc Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. © 2019 Arm Limited
Brent Gorda
September 25th, 2019
Arm in HPC

2. 2 © 2019 Arm Limited
© 2019 Arm Limited
• Parkinson’s & Osteoporosis
Ongoing research in Bristol: New Drugs ‘In Silico’
Images courtesy of Bristol University

3. 3 © 2019 Arm Limited
© 2019 Arm Limited
Multiphysics Simulations: Fluid Dynamics, Heat Diffusion, Electromagnetics
Images courtesy of Bristol University

4. 4 © 2019 Arm Limited
© 2019 Arm Limited
What is “Super” or “High Performance” Computing?
Lake Tahoe ~40 Trillion Gallons of water (4.0x10^12)
~2002 Supercomputers hit 40 Teraflops (Earth Simulator – Japan/NEC)

5. 5 © 2019 Arm Limited
© 2019 Arm Limited
What is “Super” or “High Performance” Computing?
The Great Lakes hold ~6.5 Quadrillion gallons of water (6.5x10^15)
2008 Supercomputers hit 1 Petaflop 1.0x10^15 (US IBM Roadrunner)

6. 6 © 2019 Arm Limited
Top500 systems over the past 25 years
1.00E-01
1.00E+00
1.00E+01
1.00E+02
1.00E+03
1.00E+04
1.00E+05
1.00E+06
1.00E+07
1.00E+08
1.00E+09
1.00E+10
1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016 2018
59.7 GFlop/s
422 MFlop/s
1.17 TFlop/s
149 PFlop/s
1.01 PFlop/s
1.56 EFlop/s
1 Gflop/s
1 Tflop/s
100 Mflop/s
100 Gflop/s
100 Tflop/s
10 Gflop/s
10 Tflop/s
1 Pflop/s
100 Pflop/s
10 Pflop/s
1 Eflop/s
SUM
N=1
N=500
Astra HPE/ArmEarth Simulator NEC ‘02 Jaguar AMD ‘09
Images courtesy of www.top500.org

7. 7 © 2019 Arm Limited
© 2019 Arm Limited
These are not embedded devices:
Images courtesy of Bristol University

8. 8 © 2019 Arm Limited
© 2019 Arm Limited
Mont-Blanc
The “legacy” Mont-Blanc vision
Denver, Nov 13th 2017Arm HPC User Group2
Vision: to leverage the fast growing market of mobile technology for
scientific computation, HPC and data centers.
2012 2013 2014 20162015 2017 2018
Mont-Blanc 2
Mont-Blanc 3
Early Research into the Efficacy of Arm for HPC

9. 9 © 2019 Arm Limited
© 2019 Arm Limited
Catalyst UK: Accelerating ARM Adoption in UK
Industry PartnersProgram Goals
Measures of SuccessConfigs & Timeline
– Deployment: Deployment of HPC clusters at
multiple UK sites, supported for 3-year period
providing access to academia & industry
– Adoption: Early adoption of ARM for HPC in UK;
Apollo 70 Early Ship followed by customer collab.
– Applications: Customer-driven porting and opt
– Collaboration: Leveraging the success “Project
Comanche” model of customer-centric
collaboration; but based instead on Early Ship
HPE Apollo 70 product
– Exascale: Establish foundation for Exascale collab
UK Collaborations
Intended outcomes include:
– Critical HPC apps ported and demonstrated
– ISV engagements and demonstrations
– Demonstrated performance improvements
– Publications and follow-on collaborations
– Bugs filed, fixed & up-streamed to open source
– HPE: Apollo 70, HPE Performance Software - Cluster Manager, HPE
Performance Software – Message Passing Interface
– ARM: Allinea Studio (Compiler, Libraries, Forge-DDT & MAP),
OpenHPC
– Mellanox: OFED, HPC-X, OpenMPI, OpenSHMEM, MXM, SHArP
– SuSE: SLES, OpenStack, HPC Module
– Cavium: ThunderX2 SoC, technical support
– Qualcomm: Centriq SoC, technical support (tentative)
– EPCC: WRF, OpenFOAM, Rolls
Royce Hydra opt, 2 PhD candidates
– Leicester: Data-intensive apps,
genomics, MOAB Torque, DiRAC
collab
– Bristol: VASP, CASTEP, Gromacs,
CP2K, Unified Model, Hydra, NAMD,
Oasis, NEMO, OpenIFS, CASINO,
LAMMPS
– UK Government: Dept. for Bus.,
Energy & Industrial Strategy (BEIS)
Typical for each site:
– 64 Apollo 70
– Compute Nodes:
– Cavium 32c, 2.2 GHz
– 256GB memory (16GB
DIMMs)
– IB EDR CX5 Clos
– 4096+ cores
– 6 CL4300 (tentative)
– Services/Storage:
– Qualcomm Centriq
Sep-Dec: Structure
partnership, alignment
Jan: HPE/ARM SOW
Feb: Customer SoWs,
quotations, POs
Mar: SW stack validation (3rd
Party Runtime library)
Apr: Systems build, public
announcements
May: Delivery and acceptance
HPE will deliver >12,000 cores across 3
sites; amongst the largest ARM HPC
deployments in the world
HPE Confidential
Catalyst UK

10. 10 © 2019 Arm Limited
© 2019 Arm Limited
Isambard The World’s First Arm-based Production Supercomputer

11. 11 © 2019 Arm Limited
© 2019 Arm Limited
Vanguard Astra by HPE: #156 on top500
• 2,592 HPE Apollo 70 compute nodes
• 5,184 CPUs, 145,152 cores, 2.3 PFLOPs (peak)
• Marvell ThunderX2 ARM SoC, 28 core, 2.0 GHz
• Memory per node: 128 GB (16 x 8 GB DR DIMMs)
• Aggregate capacity: 332 TB, 885 TB/s (peak)
• Mellanox IB EDR, ConnectX-5
• 112 36-port edges, 3 648-port spine
switches
• Red Hat RHEL for Arm
• HPE Apollo 4520 All–flash Lustre storage
• Storage Capacity: 403 TB (usable)
• Storage Bandwidth: 244 GB/s

12. 12 © 2019 Arm Limited
© 2019 Arm Limited
Exascale – the race underway at the high end
Projected Exascale System Dates
U.S.
▪ Sustained ES*: 2022-2023
▪ Peak ES: 2021
▪ ES Vendors: U.S.
▪ Processors: U.S. (some ARM?)
▪ Cost: $500M-$600M per system
(for early systems), plus heavy
R&D investments
52
China
▪ Sustained ES*: 2021-2022
▪ Peak ES: 2020
▪ Vendors: Chinese (multiple sites)
▪ Processors: Chinese (plus U.S.?)
▪ 13th 5-Year Plan
▪ Cost: $350-$500M per system,
plus heavy R&D
EU
▪ PEAK ES: 2023-2024
▪ Pre-ES: 2020-2022 (~$125M)
▪ Vendors: US and then European
▪ Processors: x86, ARM & RISC-V
▪ Initiatives: EuroHPC, EPI, ETP4HPC, JU
▪ Cost: Over $300M per system, plus heavy
R&D investments
Japan
▪ Sustained ES*: ~2021/2022
▪ Peak ES: Likely as a AI/ML/DL system
▪ Vendors: Japanese
▪ Processors: Japanese ARM
▪ Cost: ~$1B, this includes both 1 system
and the R&D costs
▪ They will also do many smaller size
systems
* 1 exaflops on a 64-bit real application 52© Hyperion Research

13. 13 © 2019 Arm Limited
© 2019 Arm Limited
Exascale - Fujitsu A64FX

14. 14 © 2019 Arm Limited
© 2019 Arm Limited
Exascale – European Processor Initiative
GPP AND COMMON ARCHITECTURE
9
ZEUS MPPA
eFPGA
FPGA
FPGA
ZEUS ZEUS
ZEUS ZEUS EPAC
HBM
memories
DDR
memories
PCIe gen5
links
HSL
links
D2D links
to adjacent chiplets
EPAC - EPI Accelerator (TITAN)
MPPA - Multi-Purpose Processing Array
eFPGA - embedded FPGA
Cryptographic ASIC (EU Sovereignty)

15. 15 © 2019 Arm Limited
Arm HPC Software Ecosystem
ClusterManagementTools:
Bright,HPECMU,xCat,Warewulf
Linux OS Distro of choice:
RHEL, SUSE, CENTOS,…
Arm Server Ready Platform:
Standard OS compatible FW and RAS features
HPC Applications:
Open-source, Owned, and Commercial ISV codes
Job schedulers
and Resource
Management:
SLURM, IBM LSF,
Altair PBS Pro,
etc.
Programming
Languages:
Fortran, C, C++
via
GNU, LLVM, Arm
& OEMs
Debug and
performance
analysis tools:
Arm Forge,
Rogue Wave,
TAU, etc.
Filesystems:
BeeGFS,
LUSTRE, ZFS,
HDFS, GPFS
App/ISA specific optimizations, optimized libs and intrinsics:
Arm PL, BLAS, FFTW, etc.
Communication Stacks and run-times:
Mellanox IB/OFED/HPC-X, OpenMPI, MPICH, MVAPICH2, OpenSHMEM, OpenUCX, HPE MPI
Parallelism
standards:
OpenMP
(omp / gomp),
MPI, SHMEM
(see below)
User-space
utilities,
scripting,
containers, and
other packages:
Singularity,
Openstack,
OpenHPC,
Python, NumPy,
SciPy, etc.

16. 16 © 2019 Arm Limited
Porting HPC apps to the Arm platforms
Ø The platform just works – porting in 2 days is the common experience
Build recipes online at https://gitlab.com/arm-hpc/packages/wikis/home
LAMMPS CESM2 MrBayes Bowtie
AMBER Paraview SIESTA UMNAMD
VASP MILCWRF GEANT4
Quantum
ESPRESSO
DL-Poly NEMOGAMESSOpenFOAM VisIT
QMCPACKAbinitBLAST NWCHEM BWA
GROMACS
Chem/Phys
Weather
CFD
Visualization
Genomics

17. 17 © 2019 Arm Limited
© 2018 Arm Limited
Arm in IOT
We design & license IP, we do not
manufacture chips
Partners build products for their
target markets
One size does not fit for all
HPC is a great fit for
co-design and collaboration
Partnership is key Choice is good
21 billion chips in the past year
Mobile/Embedded/IoT/
Automotive/GPUs
And now … servers
Arm Technology Connects the World

18. 18 © 2019 Arm Limited
© 2019 Arm Limited
Edge
Edge
Critical Data
Massive Amounts of Data
z
z
Edge
5G
CORTEX
HPC
Cloud
Data Centers
The New Architecture

19. 19 © 2019 Arm Limited
Confidential © 2019 Arm Limited
• Historically strong focus on high-end systems and balance:
• B:F Ratio’s of the late 1990’s thru 2010
• Parallel processing at massive scale
• Low-latency / high BW interconnects
• Citing: S/W maintenance, roll-out, cooling/power
• Workloads:
•Historical workloads scientific simulation
•Recent new workloads attracted to “high-end” capabilities of
HPC architectures: big data, Deep Learning/AI
•HPC Leads in technology acceptance (think Formula-1)
HPC is an excellent partner for the ecosystem
HPC is an Architecture

20. 20 © 2019 Arm Limited
© 2019 Arm Limited
Arm is Data driven, from the edge to the core

21. The Cloud to Edge Infrastructure Foundation
for a World of 1T Intelligent Devices
Thank You
Arm.com/hpc