High-level Meeting & Workshop on Environmental and Scientific Open Data for Sustainable Development Goals in Developing Countries. Madagascar, 4-6 December 2017

1. Building High Performance
Computing Capability in the
African Continent.
International Workshop on
Environmental and Scientific Open
Data for Sustainable Development
Goals in Developing Countries
5th September 2017

2. Background: The CHPC in SA
CHPC

3. User Community: South Africa
CHPC
Existing Users
Future Users

4. Slide #4
Background
 CHPC is national HPC facility funded by the DST and managed by
CSIR
 It is part of the National Integrated Cyber- Infrastructure System
(NICIS)
 Started operations in June 2007
 Staff complement of 40
 CHPC has Research, Technical and Operational divisions
 Provide HPC resources and services to universities, science
councils and industry
 Provide support to large scale science projects such as H3Africa,
CERN, SKA and IPCC
 Human Capital Development is core to CHPC success

5. Investment in HPC Systems
2007: IBM e1350; AMD Opteron; 2.4 GHz; 640
cores
2.5TFlops (Linpack)
2008: IBM Blue Gene/P; 0.8 GHz; 4096 c
11.5 Tflops (Linpack)
2009 : Sun Constellation
Cluster
2.9/3.0 GHz; 2684 cores
27 Tflops (Linpack)
2010: GPU Cluster; 2.4 GHz
96 cpu cores; 22 GPU’s
16 TFlops (Linpack); 14 TB
2011: Tsessebe Cluster
Upgraded 2.9GHz (4032 cores)
61.6 Tflops total (Linpack)
4 PB DIRISA
Storage
Unit
90 TB GP File System
Slide #5

6. Slide #6
System Configuration Phase 1 Phase 2
Dell PowerEdge C6320 Servers:
Standard Compute nodes 128GB (64GB) / node 1 008 1 368
2 x Intel Xeon E5-2690 v3 (Haswell) processors
(12 Cores Each  24 cores / node)
24 192 32 832
Dell PowerEdge R930 servers:
Large Memory Compute Nodes 1024GB / node (FAT nodes) 5 5
4 x Intel Xeon E7-4850 v3 processors
(14 Cores Each  56 cores / node)
280 280
Infiniband FDR 2:1 Blocking (56 Gbps)
Parallel Storage (Useable) PB 4 4
Total Number of Racks (including Compute, Login,
Management and Storage Nodes)
19 24
Centos 7.1 with Bright Cluster Manager and Altair PBS Pro
Total Linpack Performance (Tflop/s) 783 1029
The Road to PetaFLOP
3 May 2016 7 March 2017

7. Launch of the HPC system
• 127 on TOP500
• Awarded the fastest
supercomputer in Continent

8. Slide #8
Materials Science
26%
Chemistry
23%Earth Sciences
15%
Bioinformatics
12%
Health Sciences
11%
Astronomy
5%
Other
8%
Resources at the CHPC
CPU Hours:
Science Domain
Distribution

9. Slide #9
Who is using the CHPC?
# Research Programs: Research Domain Distribution (210 Programs)
Materials Science
14%
Chemistry
22%
Earth Sciences
9%Bioinformatics
17%
Health Sciences
5%
Astronomy
7%
Physics
2%
Engineering
13%
Applied
Mathematics
5%
Computer
Science
2%
Other
4%

10. Slide #10
Who is using the CHPC?
Non-Academic Public
 CSIR
 SAWS
 ARC
 NZG
 SAAO
 SANBI
 Sugar Institute
 NICD
 NECSA
SA Academic
75%
SA Public
14%
SA Industry
7%
African
Academic
4%
Research Programmes
Distribution

11. Slide #11
Student Cluster Competition (SCC):
Training undergraduate students in HPC
Three stage program:
(1) Winter school: 22 teams
(2) CHPC National Meeting: 10 Teams
(3) ISC: 1 Team representing SA
ISC’15 Runner-Up
Frankfurt
ISC’14 Champions
Leipzig
ISC’13 Champions
Leipzig
ISC’16 Champions
Frankfurt
Human Capital Development

12. Objectives
• To assist SKA partner countries in establishing functional cyber-
infrastructure components necessary to undertake science activities in
radio astronomy.
• To catalyse the development of a cadre of scientists and students
equipped with relevant skills to undertake scientific activities in radio
astronomy.
• To build institutional capacities in targeted universities, research
institutions and government departments to promote development of
cyber-infrastructure in support of radio astronomy activities in SKA
partner countries;
• To mobilise funding and technical resources for relevant cyber-
infrastructure and computational skills development in SKA partner
countries; and
• To facilitate strategic partnerships and collaborative efforts, both
regionally and globally, in support of the development of relevant cyber-
infrastructure and skills in SKA partner countries.

13. Big Data Proposal
• Partner countries will provide a host institution to
drive the developments of computing resources for
the telescopes and for other relevant applications.
• The designated host institution will fund the
operational and maintenance costs of the computing
and data storage infrastructure to agreed standards.
• South Africa and international collaborators to
provide training program in HPC and astronomy.
• Introduce HPC modules in the curriculum at
universities.

14. Model of access to HPC.
• Strategy to repurpose HPC systems
out of national production for local
processing capabilities or training
facilities.
• Develop partnership with universities
in partner countries to develop
training programs.
• Texas Advanced Computing Center,
Cambridge and CHPC provided
equipment.
• South Africa, through DST provided
some funding for additional network
and storage equipment.

15. Slide #15

16. Computing Developments: Madagascar
Contacts:
IOGA (Institute & Observatory of Geophysics)
U.Tana, U.Antsiranana, U.Toliara, U.Fianar
Training:
Sysadmin
Data Science (linux+python+HPC intro)
Intro to Astronomy (with NRAO in USA)
Projects:
Astro, Hydrology, Earth sciences, mathematics
(3 CHPC users)
Curriculum development for Astrophysics:
MSc student
(proposed post-doc placed in SA)

17. Slide #17
HPC on Common Ground
Workshop
Participants from Partner
Countries attending the HPC
“On-Common Ground”
Workshop in Salt Lake City, USA.

18. Slide #18
Preparation of HPC systems

19. Slide #19
Badise Mosesane at CERN
The CERN Summer Student Program accepts a limited
number of students from over 60 countries to spend
eight weeks at CERN. Students work side-by-side with
student-peers and scientists on some of the world’s
biggest experiments.
http://www.stem-trek.org/2017/07/28/badisa-wk4/

20. Progress on SKA Readiness
Item Comments Next stages
HPC roll-out for Phase 1
(Botswana, Zambia, Namibia,
Madagascar, Mauritius)
All countries received the
computing equipment and
storage and network
equipment.
• Continue to drive
utilisation of the systems
and ensure sustainability.
• HW Refresher through
Stampede
Training Workshops Astronomy, HPC , Materials
Science, Bioinformatics and
Climate Modelling.
Emphasize the outputs such as
students training, Flagship
projects and publications.
HPC roll-out for Phase 2
(Ghana, Kenya and
Mozambique)
Ghana received the equipment.
Only two sites (Kenya and
Mozambique) remaining.
Engagements with Kenya and
Mozambique on-going.
Deployment of Software. Active research in Material
Science and partnership with
Newton Fund projects.
CHPC, Development in Africa
with Radio Astronomy
(DARA) and Cardiff
University.
Research Cloud Research Project within SKA-
SDP continuing.
On-going preparation for
deployment of OpenStack at
all sites..

21. Slide #21
Progress in HPC Delivery
Completed
Progressing
Not Started
Infrastructure Staff Development Status of Hardware
Power &
cooling
Datacentre
ID'ed
Equipment
shipped
SysAdmin
/ Linux
Dedicated
time
SysAdmin
Course Projects
Working
Group
Operation
al
Used Needs
Support
Equipment
To
Operate
Mauritius:
University of
Mauritius
(UoM)
3 3 3 2 3 3 2 3 3 3 3
Namibia:
Namibia
University of
Science &
Technology
(NUST)
3 3 3 3 3 3 2 3 3 2 3
Namibia:
University of
Namibia
(UNAM)
3 3 3 3 3 3 2 3 3 2 3
Botswana:
University of
Botswana
(UB)
3 3 * 3 3 3 3 3 3 3 3
Zambia:
ZAMREN
3 3 2 3 3 3 3 3 3 3 3
Madagascar:
University of
Antananarivo
(IOGA)
3 3 2 2 2 2 2 2 1 1 1
Ghana::
3 3 3 2 3 2 2 3 2 1 3