These slides are from my talk at HPCDB, co-located with Supercomputing 2011 in Seattle.

1. Enabling Automated HPC /
Database Deployment via the
AppScale Hybrid Cloud Platform
Chris Bunch Chandra Krintz
HPCDB 2011 @ Seattle, WA
November 18, 2011

2. Cloud Computing
• Three tiers of abstraction:
• Infrastructure: Scalable hardware
• Platform: Scalable APIs
• Software: Scalable application

3. HPC in the Cloud
• Easy access to vast resources
• Hard to automatically configure and deploy
libraries
• Requires in-depth knowledge of each
technology involved
• Hard to get high performance on opaque cloud
• Wide range of APIs for similar services (e.g.,
compute, storage)

4. Solving these Problems
• Need a solution that is automated
• Minimize the amount of domain-specific
knowledge a scientist has to learn
• Need to automate configuration and
deployment
• Hard since the number of nodes used can
be dynamic in the cloud

5. Cloud Computing
• Infrastructure: Completely self-service
• Platform: Automated access to
certain APIs
• Software: Optimized for one piece of
software

6. APIs to Support
• To support scientists, we need to support:
• Web services
• HPC codes
• Databases
• And it must be simple

7. Introducing AppScale
• An open source implementation of the
Google App Engine APIs
• Can run over Amazon EC2 or Eucalyptus
• Configures and deploys
automatically
• Scientist only needs to specify the
number of nodes to run over

8. One Button
Deployment
•appscale-run-instances
• --max 32
• --infrastructure ec2

9. AppScale + Neptune
• A domain-specific language for running HPC
codes
• Supports codes written in:
• MPI, UPC, X10, and others
• Configures and deploys automatically
• Scientist only needs to specify the number
of nodes to run over

10. One Command
Deployment
•neptune :type => :mpi,
• :code => “/code/ring”,
• :nodes_to_use => 32,
• :output => “/output/ring”

11. Database Support
• Support for a dozen datastores, including:
• Neptune can also store to Amazon S3
• Or anything that uses the same APIs
• Completely automated

12. Hybrid Clouds
• The utilization of more than one cloud
• Fault tolerance: Run an app in multiple
clouds
• Portability: Migrate an app from one cloud
to another
• Adds another layer of complexity

13. Open Hybrid Clouds
• AppScale and Neptune both support
automated hybrid cloud
deployment
• Just provide your credentials
• Would like to expand both with capabilities
for special use cases
• Quantify the cost of each use case

14. Government Use Case
• Compute anywhere, store only in a private,
trusted cloud
Cloud 1 - EC2 East Coast Cloud 2 - Private Eucalyptus Deployment
App
Server & DB Node
Master
App
DB Node
Server

15. One DB in Each Cloud
• Compute anywhere, store in the same cloud
Cloud 1 - EC2 East Coast Cloud 2 - Private Eucalyptus Deployment
App
App
Server &
Server
Master
DB Node DB Node

16. Wrapping it Up
• Thanks to the AppScale team, especially co-
lead Navraj Chohan and advisor Chandra
Krintz
• AppScale 1.5 released in August
• Neptune 0.1.2 released last week
• Visit us at http://appscale.cs.ucsb.edu
• E-mail: cgb@cs.ucsb.edu