"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
Cloud and Grid Computing
1. Cloud and Grid Computing
Leen Blom
Centric Research & Development
June 21st, 2013
2. Cloud and Grid Computing
Part 1: Setting the scene
• What is Cloud Computing and Grid Computing
• Differences and similarities
• Using grids and clouds
Part 2: What happens today
• Grid activities in The Netherlands, Europe and Romania
• Privacy and Security in the Cloud
• The big cloud service providers
• Our vision on Cloud
3. What is Cloud and Grid?
Power grid metaphor for both terms
• Utility computing
• John McCarthy (AI, Lisp) opined in the 1960s that "computation
may someday be organized as a public utility."[link]
• Grid definition used nowadays (2002)
• “What is the Grid? A Three Point Checklist”,[link] Ian Foster lists
these primary attributes:
• Computing resources are not administered centrally.
• Open standards are used.
• Nontrivial quality of service is achieved.
• Cloud definition based on NIST (2011)
• NIST Definition of Cloud Computing
• NIST: National Institute of Standards and Technology
• Part of the U.S. Department of Commerce
• Origin unclear: cloud was used as a metaphor for the Internet
and a standardized cloud-like shape was used to denote a
network
4. Grid Computing
Grid Computing is concerned with
• “Coordinated resource sharing and problem solving in dynamic, multi-institutional
virtual organizations.” [Foster 2002]
Focus on
• The problem
• Coordination of resources shared
• Multi organizations involved
Tasks are “bigger” than resources needed
• One task to many resources
• Task split-up in parallel sub-tasks
Classes of Grid Computing
• High Performance Computing (HPC)
• Using message passing interfaces (MPI)
• Expensive
• Embarrassingly Parallel Computing (EPC)
• Or High Throughput Computing (HTC)
• Often Volunteer Computing like Boinc
5. Cloud Computing by NIST
Service Models
Public
Private
Community
Hybrid
Deployment
Models
source: http://www.nist.gov/itl/cloud/index.cfm
7. Why Cloud (and Grid)?
On-demand self-service
• A consumer can unilaterally provision computing capabilities without
requiring human interaction with each service provider.
Broad network access
• Capabilities are available over the network and accessed through standard
mechanisms
Resource pooling
• The provider’s computing resources are pooled to serve multiple consumers
• Customer generally has no control or knowledge over the exact location of
the provided resources
Rapid elasticity
• Capabilities can be elastically provisioned and released, in some cases
automatically, often appear to be unlimited and can be appropriated in any
quantity at any time.
Measured service
• Cloud systems automatically control and optimize resource use by
leveraging a metering
See also: http://arxiv.org/pdf/0901.0131
8. Cloud and Grid: similar concepts
On-demand self-service
• A consumer can unilaterally provision computing capabilities without
requiring human interaction with each service provider.
Broad network access
• Capabilities are available over the network and accessed through standard
mechanisms
Resource pooling
• The provider’s computing resources are pooled to serve multiple consumers
• Customer generally has no control or knowledge over the exact location of
the provided resources
Rapid elasticity
• Capabilities can be elastically provisioned and released, in some cases
automatically, often appear to be unlimited and can be appropriated in any
quantity at any time.
Measured service
• Cloud systems automatically control and optimize resource use by
leveraging a metering
Grid Cloud
Grid Cloud
Grid Cloud
Grid Cloud
Grid Cloud
Public
cloud only
Batch
oriented
Capabilities
must match
Depends
Commercial
grids
Public
cloud only
See also: http://arxiv.org/pdf/0901.0131
9. Cloud and Grid: different concepts
Grid
• Focus on the task at hand
• Need resources to complete
• Scientific research
• Might use dedicated hardware components
• Like GPU or storage
• Virtualization of jobs, not of programs
• Batch oriented
Cloud
• Focus on servicing multiple users (multi-tenant)
• Information workers in many cases
• Virtualization at several levels
• Infrastructure, Platform and Software
• Userinterfaces like Windows
10. Cloud fundamentals
Virtualization
Up- and downscaling
From Capex to Opex
• Prevents entry costs
• Easier to downscale
Increasing utilization
• Research of IBM/VMware
11. Cloud And Grid Computing
Cloud Computing is being integrated into Grids
Big Data, BigTable, Hadoop needs Grid techniques
• Shift from userinterface to “batchjobs”
Rendering software in the Cloud
• Offered as SaaS,
• Autodesk 360
• vSwarm (rendering for free)
• Underlying principles are Grid
Weather services
• Offered as SaaS to end users
• Meteo organizations
• Eurogrid.org
Financial services
13. Cloud: Privacy and Security
What is more secure: Private Cloud or Public Cloud?
• Private
• Pro: easier to isolate
• Con: need to keep knowledge up-to-date
• Public
• Pro: may expect top security measures
• Con: dependent on provider
What would respect privacy more: Private Cloud or Public Cloud?
• Private
• Pro: no intrusion of people outside
• Con: need to keep knowledge up-to-date
• Public
• Pro: easier to uniform policies
• Con: conflicting laws and jurisdiction in some cases
more than 77 million accounts
affected, 12 million had
unencrypted credit card numbers
National security electronic
surveillance program operated by
the United States National Security
Agency (NSA) since 2007
15. Grid examples
Old example: SETI@Home launched 1999
• Goal is to detect intelligent life outside Earth
• Uses Boinc: Open-source software for volunteer computing and grid computing
Life science Grid
• Contains a limited number of bioinformatics applications
• Free access for all
BiG Grid / SurfSara (NL)
• National e-Infrastructure for research, SurfSara
• Huygens: the national supercomputer / Lisa: the national compute cluster / GPU
cluster: a cluster equipped with GPUs / Grid: the Grid infrastructure / HPC Cloud: the
cloud computing infrastructure / Hadoop: Big Data analytics framework / DIS: Data
Ingest service / Collaboratorium: advanced visualization and presentation space /
Visualization: high-end render cluster
European Grid Infrastructure
• For scientists and researchers
Cartesius is the fastest supercomputer in the
Netherlands and eventually will be able to perform
more than one quadrillion computations per second
(1 Petaflop/s).
16. Grid examples
Rotterdam University of Applied Science (HR)
• Donates their spare capacity of 6000 desktop to Erasmus Medical
Center. Usage
• DNA-sequence analysis
• Simulating spread of infectious deseases
• MRI-scan analysis (Alzheimer)
• Technical
• HTCondor™ middleware
• Specialized workload management system for compute-intensive jobs
• Open Source provides by University of Wisconsin Madison
• HR research by their students (Bachelor)
• Fractal movie and rendering
• Security (desk cracker)
• Calculating prime numbers