Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
PEARC17: Cloud-enabling a Collaborative Research Platform: The GABBs Story
1. CLOUD-ENABLING A COLLABORATIVE
RESEARCH PLATFORM
THE GABBS STORY
Rajesh Kalyanam, Robert Campbell, Derrick Kearney, Leif Delgass, Larry
Biehl, Lan Zhao, Carolyn Ellis, Carol Song
Purdue University
2. I have historical
data on sub-
Saharan maize
yields
I’d love to use that
historical data in my
model predicting the
effect of climate
change on yields
I used the maize
yield data in my
thesis, how do I
cite it?
RESEARCH COLLABORATION LANDSCAPE
3. SCIENCE GATEWAYS
• Website for a scientific research community
• User, data, & content management
Data sharing, access control
Data processing, visualization, publication
Wiki, static web pages, ticketing system
• Research tools
Tool creation and web deployment
Optional HPC resource availability
and more…
4. GEOSPATIAL DATA ANALYSIS BUILDING BLOCKS
Data Management
Data sharing
Geospatial
metadata
extraction
Geospatial data
discovery (search)
Data publication
Visualization
Geospatial preview
Diverse data format
overlays
Visualization library
and control widgets
Hardware and
software rendering
Computation
Geospatial data
transformation
Subset extraction
Extended metadata
capture
GIS-enabled
workflows
GABBs provides these domain-specific capabilities for
geospatial science gateways
5. GABBS SYSTEM ARCHITECTURE
• Based on HUBzero
cyberinfrastructure framework
• Containerized tools are web-
accessible via VNC
• Scalable, external
compute/render servers
Support tool sessions
Provide hardware and
software rendering
• represent
GABBs enhancements to
HUBzero
Blue blocks
6. WHY CLOUD ENABLE ?
Easily scalable
Increasingly adopted
by academia and
industry
Automatable,
repeatable solutions
even for lay-users
Offload physical
resource
maintenance,
provisioning to cloud
service provider
Software packages as
a byproduct, enabling
manual installation
7. MOVING GABBS TO THE CLOUD
GPU Render Server
• OSGEarth
rendering software
• Default base maps
Data Storage Server
• iRODS server software
• PostgreSQL database
• iRODS microservices
Metadata
extraction
Hub search
indexing
Geospatial
preview
• GeoServer map server
HUBzero Web Host
• GABBs-HUBzero
integration software
• GABBs hub tools/libraries
and dependencies
MultiSpec
pyMapLib
GeoBuilder
• iRODS FUSE mount
Hub server DNS/IP
iRODS server
DNS/IP
8. MOVING GABBS TO THE CLOUD
• Utilize Amazon Web Services (AWS)
Vanilla HUBzero already available on AWS
• Provision other compute/storage servers
iRODS + GeoServer
Render server with appropriate hardware
• Setup installation/configuration of necessary
software and server interconnects
9. SOME AWS CONSIDERATIONS
• Each server (EC2 instance) requires a base machine
image (AMI)
• AMIs cannot contain hardcoded administrator
usernames, passwords
• Network security considerations
Servers require unrestricted traffic between them
• Fully automated software installation at boot-up
rather than requiring user interaction
12. CLOUDFORMATION ISSUES
INTERLEAVED DEPLOYMENT AND CONFIGURATION
HUBZERO: {
UserData: {
#!/bin/bash
/tmp/update-irods-connection.sh
{Fn::GetAtt:[IRODS,PublicDnsName]}
IRODS: {
UserData: {
#!/bin/bash
/tmp/update-hub-solr-connection.sh
{Fn::GetAtt:[HUBZERO,PublicDnsName]
}
• Resource reference forces deployment ordering
• UserData, cloud-init not separable from actual resource
provisioning
• Gives rise to cyclic dependency above
• Resource considered “created” before UserData is done
HUBzero UserData may begin execution before iRODS server user setup is
done
13. CLOUDFORMATION ISSUES
INTERLEAVED DEPLOYMENT AND CONFIGURATION
• Solving cyclic dependency
Decide on an ordering (iRODS before HUBzero)
Configure HUBzero DNS in iRODS later via HUBzero
UserData
• Ensuring UserData completion before next resource
deployment
Utilize AWS WaitCondition to force resource dependencies
WaitCondition considered “created” only when signalled
Signal WaitCondition at end of UserData script
14. CLOUDFORMATION ISSUES
SHARED PARAMETERS
HUBZERO: {
UserData: {
#!/bin/bash
/tmp/update-irods-connection.sh
{Fn::GetAtt:[IRODS,PublicDnsName]}
{"Ref": "gabbsUser"}
IRODS: {
UserData: {
#!/bin/bash
/tmp/add-irods-user.sh
{"Ref": "gabbsUser"}
• HUBzero needs to setup connection to iRODS server, after
iRODS has created a new service user
Username required during deployment of both servers
• Cannot be hardcoded into AMIs
15. CLOUDFORMATION ISSUES
SHARED PARAMETERS
• Solving shared (dynamic) parameter requirement
Use CloudFormation parameters
User prompted to fill in value
Cannot default to random value (only hardcoded defaults
supported)
Necessitates explaining (ideally behind-the-scenes)
parameters
16. CONCLUSION & FUTURE WORK
• Current CloudFormation Template:
https://mygeohub.org/groups/gabbs/release
• Employ AutoScaling to scale up render servers and
tool execution servers on demand
• Provide configuration flexibility in CloudFormation
template
• Utilize configuration management tools like Chef,
Puppet to streamline server types and software
installation
17. ACKNOWLEDGEMENTS
This work was supported by the NSF Award ACI - 1261727
CIF21 DIBBs : Integrating Geospatial Capabilities into
HUBzero