XSEDE is a major research infrastructure in the United States with collaborations worldwide supporting thousands of researchers across a wide range of domains. XSEDE has taken an integrative and holistic approach to supporting researchers in the use of the varying resources and services available via XSEDE. This presentation will breifly review XSEDE and its vision and provide a discussion of the efforts within XSEDE targeted at supporting research communities.

1. December 15, 2014
Supporting Research Communities with XSEDE
John Towns
PI and Project Director, XSEDE
Executive Director, Science & Technology, NCSA
jtowns@ncsa.illinois.edu

2. License terms
•Please cite as: Towns, John. Supporting Research Communities with XSEDE, December 2014, [http://www.slideshare.net/jtownsil/supporting-research-communities- with-xsede]
•ORCID ID: http://orcid.org/0000-0001-7961-2277
•Except where otherwise noted, by inclusion of a source URL or some other note, the contents of this presentation are © by the Board of Trustees of University of Illinois. This content is released under the Creative Commons Attribution 3.0 Unported license (http://creativecommons.org/licenses/by/3.0/). This license includes the following terms: You are free to share – copy and redistribute the material in any medium or format; and to adapt – remix, transform, and build upon the material for any purpose, even commercially.
•This can be done under the following conditions: attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
2

3. SNIC User Forum 2014
15-16 December 2014
XSEDE in Context
•XSEDE is an award made under the eXtreme Digital solicitation
–TeraGrid Phase III: eXtreme Digital Resources for Science and Engineering (XD), NSF 08-571
•Consistent with NSF’s vision and strategy statements
•NOTE: internationalization
–Cyberinfrastructure == eScience Infrastructure
3

4. SNIC User Forum 2014
15-16 December 2014
NSF’s Strategic Planning Documents
•Investing in Science, Engineering, and Education for the Nation's Future - National Science Foundation Strategic Plan for 2014-2018
•www.nsf.gov/pubs/2014/nsf14043/nsf14043.pdf
–Vision: A Nation that creates and exploits new concepts in science and engineering and provides global leadership in research and education.
•Cyberinfrastructure Framework for 21st Century Science and Engineering
•www.nsf.gov/cif21
•NSF’s Advanced Computing Infrastructure: Vision and Strategic Plan
•www.nsf.gov/pubs/2012/nsf12051/nsf12051.pdf
4

5. SNIC User Forum 2014
15-16 December 2014
Original Motivation for XSEDE
•Scientific advancement requires a variety of resources and services
–and thus availability of comprehensive cyberinfrastructure composed of heterogeneous digital resources
•Computational science better served if we leverage aggregate expertise of a small number of leading institutions
–not fully centralized at a single institution; not fully decentralized
–full centralization less agile, single point of failure
–different sites each offer a unique perspective and talent to address a particular suite of community needs,
–best to have several leadership perspectives for addressing the broad range of disciplinary needs
5

6. SNIC User Forum 2014
15-16 December 2014
XSEDE – accelerating scientific discovery
XSEDE’s Vision: a world of digitally-enabled scholars, researchers, and engineers participating in multidisciplinary collaborations while seamlessly accessing computing resources and sharing data to tackle society’s grand challenges.
XSEDE’s Mission:
to substantially enhance the productivity of a growing community of scholars, researchers, and engineers through access to advanced digital services that support open research; and to coordinate and add significant value to the leading cyberinfrastructure resources funded by the NSF and other agencies.
6

7. SNIC User Forum 2014
15-16 December 2014
Strategic Goal #1: Deepen and Extend Use
XSEDE will:
a.deepen the use—make more effective use—of the advanced digital services ecosystem by existing scholars, researchers, and engineers, and
b.extend the use to new communities.
c.We will contribute to preparation—workforce development—of the current and next generation of scholars, researchers, and engineers in the use of advanced digital services via training, education, and outreach; and
d.we will raise the general awareness of the value of advanced digital services.
7

8. SNIC User Forum 2014
15-16 December 2014
Strategic Goal #2: Advance the Ecosystem
Exploiting its internal efforts and drawing on those of others, XSEDE will advance the broader ecosystem of advanced digital services by:
a.creating an open and evolving e-infrastructure, and
b.enhancing the array of technical expertise and support services offered
8

9. SNIC User Forum 2014
15-16 December 2014
Strategic Goal #3: Sustain the Ecosystem
XSEDE will sustain the advanced digital services ecosystem by:
a.assuring and maintaining a reliable, efficient, and secure infrastructure, and
b.providing excellent user support services
c.XSEDE will further operate an effective, productive, and
d.innovative virtual organization
9

10. SNIC User Forum 2014
15-16 December 2014
XSEDE Factoids: high-order bits
•5 year, US$121M project
–plus US$9M, 5 year Technology Investigation Service
•separate award from NSF
–option for additional 5 years of funding upon major review after PY3
•No funding for major hardware
–coordinate, support and create a national/international cyberinfrastructure
–coordinate allocations, support, training and documentation for >$100M of concurrent project awards from NSF
•~112 FTE /~250 individuals funded across 20 partner institutions
–this requires solid partnering!
10

11. Total Research Funding Supported by XSEDE in Program Year 3
11
$767 million in research supported by XSEDE in PY3 (July 2013-June 2014)
Research funding only. XSEDE leverages and integrates additional infrastructure, some funded by NSF (e.g. Track 2 systems) and some not (e.g. Internet2).

12. SNIC User Forum 2014
15-16 December 2014
What is XSEDE?
•An ecosystem of advanced digital services accelerating scientific discovery
–support a growing portfolio of resources and services
•advanced computing, high-end visualization, data analysis, and other resources and services
•interoperability with other infrastructures
•A virtual organization (partnership!) providing
–dynamic distributed infrastructure
–support services and technical expertise to enable researchers engineers and scholars
•addressing the most important and challenging problems facing the nation and world
•More than just a project funded by the National Science Foundation
–XSEDE is a path-finding experiment in how to develop, deploy and support e-science infrastructure
12

13. SNIC User Forum 2014
15-16 December 2014
•World-class leadership
–partnership led by NCSA, NICS, PSC, TACC and SDSC
•CI centers with deep experience
–partners who strongly complement these CI centers with expertise in science, engineering, technology and education
XSEDE’s Distinguishing Characteristics: Governance
13

14. SNIC User Forum 2014
15-16 December 2014
Science Requires Seamlessly Integrated “Advanced Digital Services”
•Often use the terms “resources” and “services”
–these should be interpreted very broadly
–most are likely not operated by XSEDE
•Examples of resources
–compute engines: HPC, HTC (high throughput computing), campus, departmental, research group, project, …
–data: simulation output, input files, instrument data, repositories, public databases, private databases, …
–instruments: telescopes, beam lines, sensor nets, shake tables, microscopes, …
–infrastructure: local networks, wide-area networks, …
•Examples of services
–collaboration: wikis, forums, telepresence, …
–data: data transport, data management, sharing, curation, provenance, …
–access/use: authentication, authorization, accounting, …
–coordination: meta-queuing, …
–support: helpdesk, consulting, ECSS, training, …
–And many more: education, outreach, community building, …
14

15. SNIC User Forum 2014
15-16 December 2014
XSEDE offers efficient and effective integrated access to a variety of resources
•Leading-edge distributed memory systems
•Very large shared memory systems
•High throughput systems, including Open Science Grid (OSG)
•Visualization engines
•Accelerators like GPUs and Xeon PHIs
Many scientific problems have components that call for use of more than one architecture.
15

16. SNIC User Forum 2014
15-16 December 2014
XSEDEnet – Using Internet2’s AL2S

17. SNIC User Forum 2014
15-16 December 2014
XSEDE User Portal: THE User Site portal.xsede.org
•XSEDE User Portal (XUP) is designed to be the only site a user needs to use XSEDE
•XUP presents information relevant to users
–user info is easier to find
–XUP also provides dynamic data about XSEDE systems
–capabilities to manage usage, files, data
•As a user you can
–request an allocation, and manage allocations
–sign up for training
–request help
–manage file and data, and much more!
–Portal provides single sign-on to all XSEDE resources

18. SNIC User Forum 2014
15-16 December 2014
Novel and Innovative Projects: Proactive efforts in identifying subfields that are not traditional for HPC users
Goal is to enable transformative breakthroughs in fields new to HPC using the scale of XSEDE resources and services
18

19. SNIC User Forum 2014
15-16 December 2014
Mao Ye (U. of Illinois) Computational Finance
19
•Showed that by using odd lots and rapid trading, traders were able to mask what they were doing.
•His first findings contributed to a change in NASDAQ and New York Stock Exchange rules, such that all trades are reportable and visible moment-by-moment.
•Later work suggests that ever-faster trades may be destabilizing the markets.

20. SNIC User Forum 2014
15-16 December 2014
CIPRES Science Gateway for Phylogenetics
•Currently accounts for ~30% of XSEDE users
•Served 10,000th user since birth as a gateway in Dec 2009
–over 350k total job submissions
–averaging 250 new users per month/ with 1000 users total submitting 12-14,000 jobs per month
–users from 80% of EPSCoR states, significant classroom use
•~1200 publications including Science, PNAS and Cell
•“I did not know there were this many scientists out there to serve, but there is (still) no sign of usage leveling off.” – Mark Miller, PI
•“CIPRES has moved the needle of what phylogenetic scientists think is possible.” – Brent Mishler, UCB, original CIPRES member
20

21. SNIC User Forum 2014
15-16 December 2014
Direct interactions with the Community
•Facilitate broad range of ground-breaking research
–provided in-depth support contributing to improved user productivity
–supported over 12,000 publications to date
•Seamlessly integrate and retire resources
–transition users smoothly
•Pursue new disciplinary areas
–increasing the diversity of disciplines utilizing advanced digital services
•Campus Champions continue to reach new heights
–over 200 Champions at more then 165 institutions
–expanding program: Regional, Student, and Domain Champions
21

22. SNIC User Forum 2014
15-16 December 2014
XSEDE Computational User Census
22

23. SNIC User Forum 2014
15-16 December 2014
Centralized/Coordinated Services Provide Value Add
•User productivity enhancements
–XSEDE User Portal, single sign-on, allocation processes
•Centralized/coordinated support services
–coordination of problem resolution, extended support disciplinary breadth and depth
•National leadership function
•Training, Education, Outreach
–national scope
23

24. SNIC User Forum 2014
15-16 December 2014
Enhanced User Productivity Examples
•The XSEDE User Portal as the place for users to go to get information and support
–a single location for their needs
–create a single account that gives you access to all XSEDE resources: over 22,000 accounts!
•Single sign-on
–use institutional credentials to authenticate to all XSEDE resources and services
–OAuth service to allow other services to leverage XSEDE’s infrastructure
•e.g. third party authentication for a science gateway
•Same look and feel of documentation at all sites
–using a common template facilitates finding information
24

25. SNIC User Forum 2014
15-16 December 2014
Enhanced User Productivity Examples
•Single allocations process
–single request to gain access to all XSEDE-allocated resources
–expert help in selecting the right resource from the entire array of nationally-available resources of XSEDE
–11 compute resources (HPC and HTC), 2 visualization resources, 6 storage resources, VM hosting service
•https://www.xsede.org/resources/overview
•Unified tool set assures usability and reliability
–distributed team collaborates in support of various enterprise services
–data management tools, usage accounting and account management
•Support XDMoD usage analysis portal: https://xdmod.ccr.buffalo.edu/
25

26. SNIC User Forum 2014
15-16 December 2014
Centralized/coordinated support services
•Coordinated problem resolution
–field and route over 10,000 tickets annually
–work with Service Providers to resolve all reported issues
•Extended Collaborative Support Services (ECSS)
–single, coordinated effort to bring the right expertise to bear on issues raised by any user on any resource(s)
–no replication across Service Providers
•disciplinary breadth of expertise, allows coverage of domains composed of diverse sub-domains
–Novel and Innovative Projects
•support of emerging & innovative research
–optimization of widely used community codes
•prioritizing and coordinating effort
•often optimized for multiple architectures
•improving code substantially is as better than buying more hardware
26

27. Diverse ECSS Expertise Possible Because of Scale
•Fields of expertise: astrophysics, bioinformatics, CFD, chemistry, computer science, climate modeling, engineering, genomics, hydrology, humanities , machine learning, molecular dynamics, phylogenetics, physics, seismology, statistics.
•Technologies: clusters, large shared memory systems, MICs, GPUs
•Languages: C, C++,Fortran, MPI, OpenMP, Java, JavaScript, shell programming, CUDA, OpenACC, Python, R, MATLAB
•Techniques: benchmarking, cloud computing, Condor, data mining, databases, FFTs, finite element methods, grid generation, grid middleware, Lattice Boltzmann methods, libraries, linear algebra, Monte Carlo methods, parallel debugging, parallel I/O, petascale computing, scheduling, science gateways, visualization, workflows
27

28. SNIC User Forum 2014
15-16 December 2014
Training, Education, Outreach
•Single set of programs of national scope
–Training & Education
–Underrepresented Community Engagement
–Campus Champions
•Programs serve a more diverse community
–single coordinated set of programs without competition
–one consistent message and set of technical information makes it easier for technology adoption to spread organically
•Better ability to cover the entire nation in outreach:
–XSEDE Conference
–users in all 50 states, D.C., and US territories
–Campus Champions - 49 States
–XSEDE staff physically located in 18 states + DC
•Over 90,000 training registrations in past year!
–HPCU and CI-Tutor, as well as center trainings, have been used in universities around the country to prepare students to use the nation's pre-eminent computational resources
28

29. SNIC User Forum 2014
15-16 December 2014
National Leadership Functions
•Campus Bridging: integrating campuses into the national infrastructure
–aligns national CI – single entity to bridge to
–enables the national research community to better leverage non-federal investments in campus CI
–extends the value of software created by community through XSEDE-Compatible Basic Cluster distribution
•Science Gateways
–one national cyberinfrastructure environment to serve as a back end for gateways
•XSEDE becomes a coordinating point for national and international entities who want to engage the NSF HPC programs
29

30. Convenience Requirements will Always Increase
•Each generation of users requires more convenience than the former
•We must always be adding new capabilities while maintaining and extending existing reliability
•XSEDE has learned from the past
–adds value in how we address going forward
30
Change is the only Constant
– Heraclitis 535BC-475BC
No, his mind is not for rent To any god or government. Always hopeful, yet discontent, He knows changes aren't permanent, But change is.
– Rush - Tom Sawyer

33. Questions?