SlideShare ist ein Scribd-Unternehmen logo
1 von 19
Downloaden Sie, um offline zu lesen
Preparing for High-Luminosity LHC. HNsciCloud presented at the Open Science Conference 2018 in Berlin
Preparing for
High-Luminosity LHC
Bob Jones
CERN
Bob.Jones <at> cern.ch
The Mission of
CERN
 Push back the frontiers of knowledge
E.g. the secrets of the Big Bang …what was the matter like
within the first moments of the Universe’s existence?
 Develop new technologies for
accelerators and detectors
Information technology - the Web and the GRID
Medicine - diagnosis and therapy
 Train scientists and engineers of
tomorrow
 Unite people from different countries
and cultures
4
CERN: founded in 1954: 12 European States
“Science for Peace”
Today: 22 Member States
Member States: Austria, Belgium, Bulgaria, Czech Republic, Denmark, Finland,
France, Germany, Greece, Hungary, Israel, Italy, Netherlands, Norway, Poland,
Portugal, Romania, Slovak Republic, Spain, Sweden, Switzerland and
United Kingdom
Associate Member States: India, Lithuania, Pakistan, Turkey, Ukraine
States in accession to Membership: Cyprus, Serbia, Slovenia
Applications for Membership or Associate Membership:
Brazil, Croatia
Observers to Council: Japan, Russian Federation, United States of America;
European Union, JINR and UNESCO
~ 2300 staff
~ 1400 other paid personnel
~ 12500 scientific users
Budget (2016) ~1000 MCHF
A New Era in Fundamental Science
Exploration of a new energy frontier
in p-p and Pb-Pb collisions
LHC ring:
27 km circumference
CMS
ALICE
LHCb
ATLAS
The Worldwide LHC Computing Grid
Tier-1:
permanent storage,
re-processing,
analysis
Tier-0 (CERN): data
recording,
reconstruction and
distribution
Tier-2:
Simulation,
end-user analysis
2 million jobs/day
700 PB of storage
nearly 170 sites,
40+ countries
WLCG:
An International collaboration to distribute and analyse LHC data
Integrates computer centres worldwide that provide computing and storage
resource into a single infrastructure accessible by all LHC physicists 6
DocumentClassification:Restricted
WLCG Data 2016-17
Frédéric Hemmer, 1 December 2017 7
2016: 49.4 PB LHC data/
58 PB all experiments/
73 PB total
2017: ~37 PB (30 Nov)
ALICE: 4.4 PB
ATLAS: 17.4PB
CMS: 10.7 PB
LHCb: 4.8 PB
220 PB on tape
550 M files
Slide 8
Open Data at CERN
• The 4 main LHC experiments have approved
Open access policies whereby (increasing)
fractions of their data are made available after
suitable “embargo periods”
• These refer to “derived data” + documentation + s/w
and environment
• But LHC data volume is already >200PB
• Expected to reach ~10(-100)EB during HL-LHC
• We need to preserve all of this (but not all is Open)
Jamie Shiers (CERN)
9
Slide 10
Slide 11
CERN as a Trusted Digital Repository
• We believe ISO 16363 certification will allow us
to implement best practices and ensured for the
long-term.
Jamie Shiers (CERN)
• Scope: Scientific Data and
CERN’s Digital Memory
• Timescale: complete prior
to 2020
Artefactual Systems
Slide 12
Challenges: LHC Run3 and Run4 Scale
Raw data volume for LHC
increases exponentially and with it
processing and analysis load
Technology at ~20%/year will bring
x6-10 in 10-11 years
Estimates of resource needs at HL-
LHC x10 above what is realistic to
expect from technology with
reasonably constant cost
First run LS1 Second run LS2 Third run LS3 HL-LHC
…
FCC?
2009 2013 2014 2015 2017 201820112010 2012 2019 2023 2024 2030?20212020 2022 …
0
100
200
300
400
500
600
700
800
900
1000
Raw Derived
Data estimates for 1st year of HL-LHC (PB)
ALICE ATLAS CMS LHCb
0
50000
100000
150000
200000
250000
CPU (HS06)
CPU Needs for 1st Year of HL-LHC (kHS06)
ALICE ATLAS CMS LHCb
2025
CPU:
• x60 from 2016
Data:
• Raw 2016: 50 PB  2027: 600 PB
• Derived (1 copy): 2016: 80 PB  2027: 900 PBTechnology revolutions are needed
2016
13
Evolution of Computing – Community White Paper*
14
The Hybrid Cloud Model
Brings together
• research organisations,
• data providers,
• publicly funded e-
infrastructures,
• commercial cloud service
providers
In a hybrid cloud with
procurement and governance
approaches suitable for the
dynamic cloud market In-house
Helix Nebula Science Cloud
Joint Pre-Commercial Procurement
Bob Jones, CERN 15
Procurers: CERN, CNRS, DESY, EMBL-EBI, ESRF, IFAE,
INFN, KIT, STFC, SURFSara
Experts: Trust-IT & EGI.eu
Resulting IaaS level services support use-cases from
many research communities
Deployed in a hybrid cloud combining procurers data
centres, commercial cloud service providers, GEANT
network and eduGAIN fed. identity mgmt.
Co-funded via H2020 Grant Agreement 687614
Total procurement budget >5.3M€
Challenges
Innovative IaaS level cloud services integrated with procurers in-
house resources and public e-infrastructure to support a range
of scientific workloads
Compute and Storage
support a range of virtual machine and container configurations including HPC
working with datasets in the petabyte range accessible transparently
Network Connectivity and Federated Identity Management
provide high-end network capacity via GEANT for the whole platform with
common identity and access management
Service Payment Models
explore a range of purchasing options to determine those most appropriate
for the scientific application workloads to be deployed
Bob Jones, CERN 16
The Pre-Commercial Procurement process
15/03/2018
EOSC
See www.HNSciCloud.eu
Slide 19
Two questions to conclude this talk
Highlighting some of the issues that service
providers face as they move towards EOSC
The Zenodo service is developed as a marginal activity
by CERN and relies on the lab’s internal IT infrastructure
Demand continues to grow and CERN has defined a
quota which limits the storage capacity per upload
Accelerating Science and Innovation

Weitere ähnliche Inhalte

Mehr von Helix Nebula The Science Cloud

This Helix Nebula Science Cloud Pilot Phase Open Session
This Helix Nebula Science Cloud Pilot Phase Open SessionThis Helix Nebula Science Cloud Pilot Phase Open Session
This Helix Nebula Science Cloud Pilot Phase Open SessionHelix Nebula The Science Cloud
 
Cloud Services for Education - HNSciCloud applied to the UP2U project
Cloud Services for Education - HNSciCloud applied to the UP2U projectCloud Services for Education - HNSciCloud applied to the UP2U project
Cloud Services for Education - HNSciCloud applied to the UP2U projectHelix Nebula The Science Cloud
 
Network experiences with Public Cloud Services @ TNC2017
Network experiences with Public Cloud Services @ TNC2017Network experiences with Public Cloud Services @ TNC2017
Network experiences with Public Cloud Services @ TNC2017Helix Nebula The Science Cloud
 
Helix Nebula Science Cloud Pilot Phase, 6 February 2018, Bologna, Italy
Helix Nebula Science Cloud Pilot Phase, 6 February 2018, Bologna, ItalyHelix Nebula Science Cloud Pilot Phase, 6 February 2018, Bologna, Italy
Helix Nebula Science Cloud Pilot Phase, 6 February 2018, Bologna, ItalyHelix Nebula The Science Cloud
 
Pilot phase Award Ceremony - INFN Introduction and welcome
Pilot phase Award Ceremony - INFN Introduction and welcomePilot phase Award Ceremony - INFN Introduction and welcome
Pilot phase Award Ceremony - INFN Introduction and welcomeHelix Nebula The Science Cloud
 
Early adopter group and closing of webinar - João Fernandes (CERN)
Early adopter group and closing of webinar - João Fernandes (CERN)Early adopter group and closing of webinar - João Fernandes (CERN)
Early adopter group and closing of webinar - João Fernandes (CERN)Helix Nebula The Science Cloud
 
Using Docker containers for scientific environments - on-premises and in the ...
Using Docker containers for scientific environments - on-premises and in the ...Using Docker containers for scientific environments - on-premises and in the ...
Using Docker containers for scientific environments - on-premises and in the ...Helix Nebula The Science Cloud
 
CERN & Huawei collaboration to improve OpenStack for running large scale scie...
CERN & Huawei collaboration to improve OpenStack for running large scale scie...CERN & Huawei collaboration to improve OpenStack for running large scale scie...
CERN & Huawei collaboration to improve OpenStack for running large scale scie...Helix Nebula The Science Cloud
 
HNSciCloud as the functional basis for the future EOSC - HNSciCloud at Open S...
HNSciCloud as the functional basis for the future EOSC - HNSciCloud at Open S...HNSciCloud as the functional basis for the future EOSC - HNSciCloud at Open S...
HNSciCloud as the functional basis for the future EOSC - HNSciCloud at Open S...Helix Nebula The Science Cloud
 
HNSciCloud represented at HUAWEI CONNECT 2017 in Shanghai
HNSciCloud represented at HUAWEI CONNECT 2017 in ShanghaiHNSciCloud represented at HUAWEI CONNECT 2017 in Shanghai
HNSciCloud represented at HUAWEI CONNECT 2017 in ShanghaiHelix Nebula The Science Cloud
 
Joao Fernandes - 6th International LSDMA Symposium, 29th August 2017
Joao Fernandes - 6th International LSDMA Symposium, 29th August 2017Joao Fernandes - 6th International LSDMA Symposium, 29th August 2017
Joao Fernandes - 6th International LSDMA Symposium, 29th August 2017Helix Nebula The Science Cloud
 

Mehr von Helix Nebula The Science Cloud (20)

Hybrid cloud for science
Hybrid cloud for scienceHybrid cloud for science
Hybrid cloud for science
 
HNSciCloud PILOT PLATFORM OVERVIEW
HNSciCloud PILOT PLATFORM OVERVIEWHNSciCloud PILOT PLATFORM OVERVIEW
HNSciCloud PILOT PLATFORM OVERVIEW
 
HNSciCloud Overview
HNSciCloud Overview HNSciCloud Overview
HNSciCloud Overview
 
This Helix Nebula Science Cloud Pilot Phase Open Session
This Helix Nebula Science Cloud Pilot Phase Open SessionThis Helix Nebula Science Cloud Pilot Phase Open Session
This Helix Nebula Science Cloud Pilot Phase Open Session
 
Cloud Services for Education - HNSciCloud applied to the UP2U project
Cloud Services for Education - HNSciCloud applied to the UP2U projectCloud Services for Education - HNSciCloud applied to the UP2U project
Cloud Services for Education - HNSciCloud applied to the UP2U project
 
Network experiences with Public Cloud Services @ TNC2017
Network experiences with Public Cloud Services @ TNC2017Network experiences with Public Cloud Services @ TNC2017
Network experiences with Public Cloud Services @ TNC2017
 
EOSC in practice - Silvana Muscella (chair EOSC HLEG)
EOSC in practice - Silvana Muscella (chair EOSC HLEG)EOSC in practice - Silvana Muscella (chair EOSC HLEG)
EOSC in practice - Silvana Muscella (chair EOSC HLEG)
 
Helix Nebula Science Cloud Pilot Phase, 6 February 2018, Bologna, Italy
Helix Nebula Science Cloud Pilot Phase, 6 February 2018, Bologna, ItalyHelix Nebula Science Cloud Pilot Phase, 6 February 2018, Bologna, Italy
Helix Nebula Science Cloud Pilot Phase, 6 February 2018, Bologna, Italy
 
Pilot phase Award Ceremony - INFN Introduction and welcome
Pilot phase Award Ceremony - INFN Introduction and welcomePilot phase Award Ceremony - INFN Introduction and welcome
Pilot phase Award Ceremony - INFN Introduction and welcome
 
Early adopter group and closing of webinar - João Fernandes (CERN)
Early adopter group and closing of webinar - João Fernandes (CERN)Early adopter group and closing of webinar - João Fernandes (CERN)
Early adopter group and closing of webinar - João Fernandes (CERN)
 
HNSciCloud pilot phase - Andrea Chierici (INFN)
HNSciCloud pilot phase - Andrea Chierici (INFN)HNSciCloud pilot phase - Andrea Chierici (INFN)
HNSciCloud pilot phase - Andrea Chierici (INFN)
 
Pilot phase Award Ceremony - T-Systems
Pilot phase Award Ceremony - T-SystemsPilot phase Award Ceremony - T-Systems
Pilot phase Award Ceremony - T-Systems
 
Pilot phase Award Ceremony - RHEA
Pilot phase Award Ceremony - RHEAPilot phase Award Ceremony - RHEA
Pilot phase Award Ceremony - RHEA
 
Overview of HNSciCloud - Bob Jones (CERN)
Overview of HNSciCloud - Bob Jones (CERN)Overview of HNSciCloud - Bob Jones (CERN)
Overview of HNSciCloud - Bob Jones (CERN)
 
Using Docker containers for scientific environments - on-premises and in the ...
Using Docker containers for scientific environments - on-premises and in the ...Using Docker containers for scientific environments - on-premises and in the ...
Using Docker containers for scientific environments - on-premises and in the ...
 
CERN & Huawei collaboration to improve OpenStack for running large scale scie...
CERN & Huawei collaboration to improve OpenStack for running large scale scie...CERN & Huawei collaboration to improve OpenStack for running large scale scie...
CERN & Huawei collaboration to improve OpenStack for running large scale scie...
 
HNSciCloud as the functional basis for the future EOSC - HNSciCloud at Open S...
HNSciCloud as the functional basis for the future EOSC - HNSciCloud at Open S...HNSciCloud as the functional basis for the future EOSC - HNSciCloud at Open S...
HNSciCloud as the functional basis for the future EOSC - HNSciCloud at Open S...
 
HNSciCloud represented at HUAWEI CONNECT 2017 in Shanghai
HNSciCloud represented at HUAWEI CONNECT 2017 in ShanghaiHNSciCloud represented at HUAWEI CONNECT 2017 in Shanghai
HNSciCloud represented at HUAWEI CONNECT 2017 in Shanghai
 
Joao Fernandes - 6th International LSDMA Symposium, 29th August 2017
Joao Fernandes - 6th International LSDMA Symposium, 29th August 2017Joao Fernandes - 6th International LSDMA Symposium, 29th August 2017
Joao Fernandes - 6th International LSDMA Symposium, 29th August 2017
 
The Science Cloud Users: Challenges and Needs
The Science Cloud Users: Challenges and NeedsThe Science Cloud Users: Challenges and Needs
The Science Cloud Users: Challenges and Needs
 

Kürzlich hochgeladen

activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 

Kürzlich hochgeladen (20)

activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 

Preparing for High-Luminosity LHC. HNsciCloud presented at the Open Science Conference 2018 in Berlin

  • 2. Preparing for High-Luminosity LHC Bob Jones CERN Bob.Jones <at> cern.ch
  • 3. The Mission of CERN  Push back the frontiers of knowledge E.g. the secrets of the Big Bang …what was the matter like within the first moments of the Universe’s existence?  Develop new technologies for accelerators and detectors Information technology - the Web and the GRID Medicine - diagnosis and therapy  Train scientists and engineers of tomorrow  Unite people from different countries and cultures
  • 4. 4 CERN: founded in 1954: 12 European States “Science for Peace” Today: 22 Member States Member States: Austria, Belgium, Bulgaria, Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Israel, Italy, Netherlands, Norway, Poland, Portugal, Romania, Slovak Republic, Spain, Sweden, Switzerland and United Kingdom Associate Member States: India, Lithuania, Pakistan, Turkey, Ukraine States in accession to Membership: Cyprus, Serbia, Slovenia Applications for Membership or Associate Membership: Brazil, Croatia Observers to Council: Japan, Russian Federation, United States of America; European Union, JINR and UNESCO ~ 2300 staff ~ 1400 other paid personnel ~ 12500 scientific users Budget (2016) ~1000 MCHF
  • 5. A New Era in Fundamental Science Exploration of a new energy frontier in p-p and Pb-Pb collisions LHC ring: 27 km circumference CMS ALICE LHCb ATLAS
  • 6. The Worldwide LHC Computing Grid Tier-1: permanent storage, re-processing, analysis Tier-0 (CERN): data recording, reconstruction and distribution Tier-2: Simulation, end-user analysis 2 million jobs/day 700 PB of storage nearly 170 sites, 40+ countries WLCG: An International collaboration to distribute and analyse LHC data Integrates computer centres worldwide that provide computing and storage resource into a single infrastructure accessible by all LHC physicists 6
  • 7. DocumentClassification:Restricted WLCG Data 2016-17 Frédéric Hemmer, 1 December 2017 7 2016: 49.4 PB LHC data/ 58 PB all experiments/ 73 PB total 2017: ~37 PB (30 Nov) ALICE: 4.4 PB ATLAS: 17.4PB CMS: 10.7 PB LHCb: 4.8 PB 220 PB on tape 550 M files
  • 8. Slide 8 Open Data at CERN • The 4 main LHC experiments have approved Open access policies whereby (increasing) fractions of their data are made available after suitable “embargo periods” • These refer to “derived data” + documentation + s/w and environment • But LHC data volume is already >200PB • Expected to reach ~10(-100)EB during HL-LHC • We need to preserve all of this (but not all is Open) Jamie Shiers (CERN)
  • 9. 9
  • 11. Slide 11 CERN as a Trusted Digital Repository • We believe ISO 16363 certification will allow us to implement best practices and ensured for the long-term. Jamie Shiers (CERN) • Scope: Scientific Data and CERN’s Digital Memory • Timescale: complete prior to 2020 Artefactual Systems
  • 12. Slide 12 Challenges: LHC Run3 and Run4 Scale Raw data volume for LHC increases exponentially and with it processing and analysis load Technology at ~20%/year will bring x6-10 in 10-11 years Estimates of resource needs at HL- LHC x10 above what is realistic to expect from technology with reasonably constant cost First run LS1 Second run LS2 Third run LS3 HL-LHC … FCC? 2009 2013 2014 2015 2017 201820112010 2012 2019 2023 2024 2030?20212020 2022 … 0 100 200 300 400 500 600 700 800 900 1000 Raw Derived Data estimates for 1st year of HL-LHC (PB) ALICE ATLAS CMS LHCb 0 50000 100000 150000 200000 250000 CPU (HS06) CPU Needs for 1st Year of HL-LHC (kHS06) ALICE ATLAS CMS LHCb 2025 CPU: • x60 from 2016 Data: • Raw 2016: 50 PB  2027: 600 PB • Derived (1 copy): 2016: 80 PB  2027: 900 PBTechnology revolutions are needed 2016
  • 13. 13 Evolution of Computing – Community White Paper*
  • 14. 14 The Hybrid Cloud Model Brings together • research organisations, • data providers, • publicly funded e- infrastructures, • commercial cloud service providers In a hybrid cloud with procurement and governance approaches suitable for the dynamic cloud market In-house
  • 15. Helix Nebula Science Cloud Joint Pre-Commercial Procurement Bob Jones, CERN 15 Procurers: CERN, CNRS, DESY, EMBL-EBI, ESRF, IFAE, INFN, KIT, STFC, SURFSara Experts: Trust-IT & EGI.eu Resulting IaaS level services support use-cases from many research communities Deployed in a hybrid cloud combining procurers data centres, commercial cloud service providers, GEANT network and eduGAIN fed. identity mgmt. Co-funded via H2020 Grant Agreement 687614 Total procurement budget >5.3M€
  • 16. Challenges Innovative IaaS level cloud services integrated with procurers in- house resources and public e-infrastructure to support a range of scientific workloads Compute and Storage support a range of virtual machine and container configurations including HPC working with datasets in the petabyte range accessible transparently Network Connectivity and Federated Identity Management provide high-end network capacity via GEANT for the whole platform with common identity and access management Service Payment Models explore a range of purchasing options to determine those most appropriate for the scientific application workloads to be deployed Bob Jones, CERN 16
  • 17. The Pre-Commercial Procurement process 15/03/2018 EOSC See www.HNSciCloud.eu
  • 18. Slide 19 Two questions to conclude this talk Highlighting some of the issues that service providers face as they move towards EOSC The Zenodo service is developed as a marginal activity by CERN and relies on the lab’s internal IT infrastructure Demand continues to grow and CERN has defined a quota which limits the storage capacity per upload