Overview and introductory remarks for the OGF sessions held May 21-22, 2015 co-located with the European Grid Initiative 2015 conference that took place the week of May 18-22, 2015 in Lisbon, Portugal. For details, see https://www.ogf.org/ogf/doku.php/events/ogf-44

1. ©2015 Open Grid Forum
The Open Grid Forum:
History, Introduction and Process
Alan Sill
VP of Standards, OGF
Director, NSF Cloud and Autonomic Computing Center
Senior Scientist, High Performance Computing Center
Adjunct Professor of Physics
Texas Tech University
1
Open Grid Forum 44, May 21-22, 2015
EGI Conference, Lisbon, Portugal

2. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
About the Open Grid Forum:
Open Grid Forum (OGF) is a global organization operating
in the areas of cloud, grid and related forms of advanced
distributed computing.
The OGF community pursues these topics through an open
process for development, creation and promotion of
relevant specifications and use cases.
OGF actively engages partners and participants throughout
the international arena through an open forum with open
processes to champion architectural blueprints related to
cloud and grid computing.
The resulting specifications and standards enable
pervasive adoption of advanced distributed computing
techniques for business and research worldwide.
2

3. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
History and Background
• Began in 2001 as an organization to promote the
advancement of distributed computing worldwide.
• Grid Forum --> Global Grid Forum --> GGF + Enterprise
Grid Alliance --> formation of OGF in 2005.
• Mandate is to take on all forms of distributed computing
and to work to promote cooperation, information exchange,
and best practices in use and standardization.
• OGF is best known for a series of important computing,
security and network standards that form the basis for
major science and business-based distributed computing
(BES, GridFTP, DRMAA, JSDL, RNS, GLUE, UR, etc.).
• We also develop cloud, networking and data standards
(OCCI, DFDL, WS-Agreement, NSI/NML, etc.) in wide use.
• Cooperative work agreements with other SDOs in place.
3

4. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
OGF Standards
OGF has an extensive set of applicable standards related to
advanced distributed grid and cloud computing and
associated storage management and network operation:
- Managing the Trust Eco-System (CA operations, AuthN/AuthZ)
- Job Submission and Workflow Management (JSDL, BES)
- Network Management (NSI, NML, NMC, NM)
- Federated Identity Management (FedSec-CG)
- Virtual Organizations (VOMS)
- Secure, fast multi--party data transfer (GridFTP, SRM)
- Service Agreements (WS-Agreement, WS-Agreement Negotiation)
- Data Format Description (DFDL)
- Cloud Computing interfaces (OCCI)
- Distributed resource management APIs (DRMAA, SAGA, etc.)
- Firewall Traversal (FiTP)
- (Many others under development)
4

5. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
• DRMAA: Distributed Resource Management Application API
Grid Engine (Univa), Open Grid Scheduler: (open source); TORQUE and
related products: Adaptive Computing; PBS Works: Altair Engineering;
Gridway: DSA Research; HTCondor: U. of Wisconsin / Red Hat;
• OGSA® Basic Execution Service Version 1.0 and BES HPC Profile:
BES++ for LSF/SGE/PBS: Platform Computing; Windows HPC Server
2008: Microsoft Corporation; PBS Works - (client only): Altair Engineering;
• JSDL: Job Submission Description Language (family of specifications):
BES++ for LSF/SGE/PBS and Platform LSF: Platform Computing;
Windows HPC Server: Microsoft Corporation; PBS Works and PBS/Pro:
Altair Engineering; Tivoli Workload Scheduler: IBM Corporation;
• WS-Agreement (family of specifications):
ElasticLM License-as-a-Service: ElasticLM; BEinGrid SLA Negotiator,
LM-Architecture and Framework: (Multiple partners); BREIN SLA
Management Framework: (Multiple partners); WSAG4J, Web Services
Agreement for Java (framework implementation): Fraunhofer SCAI.
5
OGF HPC Standards In Use
In Industry:

6. http://occi-wg.org

7. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
Starting Point: OGF Documents
7
http://ogf.org/documents

8. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
Public Comment process
8
http://redmine.ogf.org/projects/editor-pubcom/boards/

9. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
OGF Document Types
• Informational: To inform the community about a useful idea
or set of ideas.
• Experimental: To inform the community about a useful
experiment, testbed or implementation of idea or set of ideas.
• Community Practice: To inform the community of common
practice or process, with the objective to influence the
community and/or document its current practices.
• Recommendations: To publish a specification, analogous to
an Internet Standards track document. Recommendations
are initially designated as "proposed," and following further
experience and review may become full recommendations.
• Further information including guidance and advice contained
in GFD.152 at: http://ogf.org/documents/GFD.152.pdf
9

10. www.egi.euEGI-InSPIRE RI-261323 www.egi.euEGI-­‐InSPIRE
RI-­‐261323
EGI international presence
Storage Value
(yearly
increase)
Disk
(PB)
235
PB
(+69%)
Tape
(PB)
176
PB
(+32%)
Value
(yearly
increase)
CPU
cores 361,300
across
53
countries
(1.44
M
job/day)
Standards-based international collaboration

11. EGI Federated Cloud: A successful standards-based
international federated cloud infrastructure
Credit: David Wallom
Chair EGI Federated Cloud Task Force
Members
•70 individuals
•40 institutions
•13 countries
Stakeholders
•23 Resource Providers
•10 Technology Providers
•7 User Communities
•4 Liaisons
Technologies
•OpenStack
•OpenNebula
•StratusLab
•CloudStack (in
evaluation)
•Synnefo
•WNoDeS
BSC
CNRS
LMU
OeRC
Masaryk
TUD
IFAE
Cyfronet
100%IT
CESNET
RADICAL
SRCE
DANTE
FZJ
GRNET
GWDG
STFC
SARA
KTH
INFNFCTSG
EGI.eu
Imperial
CESGACETA
IFCA
IGI
IPHC
IN2P3
SZTAKI
IISAS SixSq
Standards
•OCCI (control)
•OVF (images)
•X.509 (authN)
•CDMI (storage - under
development)
(Updated July 2014)

12. www.egi.euEGI-InSPIRE RI-261323
Federated Cloud architecture
Open to new members:
Join as user, or as an IaaS/PaaS/SaaS service provider:
http://go.egi.eu/cloud
Cloud
hypervisor
(e.g.
OpenStack,
OpenNebula,
EmotiveCloud,
Okeanos…)
Cloud
site
academic/commercial
Domain
specific
services
in
Virtual
Machine
Images
FedCloud
User
interfaces
Standards
used
to
enable
federation
• OCCI:
VM
Image
management
• OVF:
VM
Image
format
• X509:
Authentication
• (CDMI:
Storage)
FedCloud
Operation
interfaces
• Information
system
(BDII)
• Monitoring
(SAM)
• Accounting
(APEL)
• AAI
(Perun)
Virtual
organisations
• GLUE2:
Resource
discovery
and
Description
• Others
in
development
Federation
monitoring

13. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
Example: Worldwide LHC
Computing Grid
13
~450,000 cpu cores
~430 Pb storage
Typical data transfer
rate: ~12 GByte/sec
Total worldwide grid
capacity: ~2x WLCG
across all grids and
VOs

14. The Role of Standards for Risk Reduction and
Inter-operation in XSEDE
XSEDE: The Next Generation of
US National Supercomputing
Infrastructure
Cloud and grid standards
now power some of the
largest academic
supercomputing
infrastructures in the
world!

15. LSN-MAGIC Meeting
February 22, 2012
XSEDE Services Layer:
Simple services combined in many ways
15
–Resource
Namespace
Service
1.1
–OGSA
Basic
ExecuNon
Service
–OGSA
WSRF
BP
–
metadata
and
noNficaNon
–OGSA-­‐ByteIO
–GridFTP
–JSDL,
BES,
BES
HPC
Profile
–WS
Trust
Secure
Token
Services
–WSI
BSP
for
transport
of
credenNals
–…
(more
than
we
have
room
to
cover
here)
Examples – (not
a complete list)
Basic message: XSEDE represents best-of-breed engagement
of open computing standards with the US cyberinfrastructure.

16. Blacklight
Shared
Memory
4k
Xeon
cores
Darter
24k
cores
Nautilus
Visualization
Data
Analytics
Keeneland
CPU/GPGPU
Stampede
460K
cores
w.
Xeon
Phi
>1000
users
Upgrade
in
2015
Wrangler
Data
Analytics
Trestles
IO-­‐intensive
10k
cores
160
GB
SSD/Flash
Gordon
Data
intensive
64
TB
memory
300
TB
Flash
Mem
Open
Science
Grid
High
throughput
124
sitesBlue
Waters
Leadership
SuperMIC
380
nodes
–
1PF
(Ivy
bridge,
Xeon
Phi,
GPU)
Over
13
million
service
units/day
typically
delivered
as
of
2014
across
all
XSEDE
supercomputing
sites
(about
3
million
core
hours/day),
totaling
about
1.6
billion
core
hours
per
year
Yellowstone
Geosciences
US National
Cyberinfrastructure
Promote an
open, robust,
collaborative,
and innovative
ecosystem
Adopt,
create and
disseminat
e
knowledge
Extend
the impact
of cyber-
infrastructure
Prepare
the current
and next
generation
Provide
technical
expertise and
support
services
Collaborate
with other CI
groups and
projects
FutureGrid
*
Maverick
Visualization
Data
Analytics
Comet
“Long
Tail
Science”
47k
cores/2
PF
High
throughput
ACI-­‐REF
Campus
sharing,
NSF
Cloud
(shared)
Grids
Credit: Irene
Qualters, US
National Science
Foundation

17. LSN-MAGIC Meeting
February 22, 2012Why Open Standards?
• Risk
reducMon
• Best-­‐of-­‐breed
mix-­‐and-­‐match
• Allows
innovaMon/compeMMon
at
more
interesMng
layers
• Facilitates
interoperaMon
with
other
infrastructures
Andrew Grimshaw
17
Takeaway message
• The
use
of
standards
permits
XSEDE
to
interoperate
with
other
infrastructures,
reduces
risks
including
vendor
lock-­‐in,
and
allows
us
to
focus
on
higher
level
capabiliMes
and
less
on
the
mundane

18. Distributed Across 124 Sites
Open Science Grid currently consists of over 124 geographical sites,
operating on a wide variety of computing systems
600k - 800k jobs/day!

19. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015 19
OGF Cooperative Agreements
In Place as of May 2015
OGF and IEEE:
• OGF co-sponsors activities at many IEEE conferences;
pursuing engagements with P2301 & P2302 working groups.
OGF and DMTF:
• Joint work register with DMTF on comparison of the OCCI and
CIMI specifications.
OGF and ISO/IEC:
• OGF has a Category A liaison with ISO/IEC JTC1 SC38 on
Cloud Computing and is working on several joint activities.
OGF and SNIA (CDMI):
• OGF has cooperative agreement w/ SNIA with respect to
CDMI and has co-hosted 16 community Cloud Interoperability
Plugfests so far (series continues at this conference!).

20. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015 20
OGF and ETSI:
• Cooperative MoU in place; contributing to ETSI CSC effort.
ETSI has joined the Cloud Plugfest series as a co-sponsor.
OGF and ITU-T:
• OGF formal liaison in place with ITU-T JCA Cloud group.
OGF and TM Forum:
• MoU in place; ongoing cross-SDO work on on End-to-End
Management of Cloud Service Agreements, including SLAs.
OGF and CSA:
• Cooperative agreement between OGF and CSA in place since
early 2012.
OGF Cooperative Agreements
In Place as of May 2015

21. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015 21
OGF and OCEAN:
• OCEAN has joined the Cloud Plugfest series as a co-sponsor
and as one of the primary sponsors of the previous
September 2013 OGF/EGI Cloud Interoperability Week.
OGF and OW2:
• OW2 has co-sponsored Cloud Interoperability Plugfests as a
formal co-sponsor and active supporter of open source cloud
projects, and supports the OCCIWare project.
OGF and XSEDE:
• Extensive XSEDE adoption of OGF standards in production.
OGF and EGI:
• Extensive community involvement, interaction and adoption of
OGF specs.
OGF Cooperative Agreements
In Place as of May 2015

22. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
Continuing series…
Oriented towards REAL
DEVELOPMENT AND TESTING
Past events
co-sponsored by many open
source and standards-related
organizations including:
OGF, CSCC, DMTF, SNIA,
OASIS, ETSI, OCEAN and OW2
Long-running
developer-oriented
in-person
standards and
software testing
series
Easy to get
involved and join
in events as
developers or
project
researchers.
Cloud Plugfest 17
at this conference!
http://cloudplugfest.org
Cloud Plugfest Developer Series:
22
More events in
planning pipeline.

23. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015
Conclusions and Future:
• OGF actively engages many partners and participants
throughout the international arena through an open forum
with open processes to promote best practices and
standards in advanced distributed computing.
• OGF occupies an important role in standards and
software development with significant uptake in advanced
distributed computing, including cloud, grid, networking
and large-scale data processing, transfer and handling.
• OGF documents support a variety of flexible architectures
for advanced scientific, community and business uses.
• OGF’s experience has enabled distributed computing built
on these architectures to provide more flexible, efficient
and utility-like global infrastructures. Join us!
23