Presentation of OpenStack Solum which provides application lifecycle manament/PaaS functionality natively in OpenStack ecosystem

1. Solum
Application Lifecycle Management
for OpenStack
Devdatta Kulkarni,
devdatta.kulkarni@rackspace.com
(irc: devkulkarni)

2. What is Solum?
OpenStack project that provides easy mechanism for
application developers to deploy and run their applications on
OpenStack starting from application’s source code
Why should you care? (the value proposition)
– For operators, Solum provides ability to make your OpenStack cloud
more useful for your clouds’ application developers by simplifying the
process of deploying applications to it
– For developers, Solum provides an easy-to-use platform for building,
testing, and deploying applications on OpenStack clouds

3. Outline
• Team introduction
• Project timeline
• Community involvement
• Project details
• Solum concepts and features
• Rackspace experience
• Upcoming features
• Call to action
• Question and answers

4. Team introduction
• Devdatta Kulkarni – PTL for Mitaka
– First member on the Solum development team
– With Rackspace for close to six years
– PhD in CS from University of Minnesota Minneapolis
• Adrian Otto – PTL till Liberty
• Key team members:
– Roshan Agrawal, Murali Allada, Ed Cranford, James Li, Melissa Kam, Dimitry
Ushakov, Angus Salkeld, Paul Czarkowski, Arati Mahimane, Paul Montgomery,
Noorul Islam, Julien Vey, Pierre Padrixe, Georgy Okrokvertskhov, Ravi Sankar
Penta, Nick Silkey, Randall Burt, Vijendar Komalla, Keith Bray, Gil Pilz

5. Timeline
October 2013: Project announced on OS-dev mailing list as a
OpenStack-related project (i.e. stackforge)
December 2013: SFO mid-cycle meet-up at Rackspace office
March 2014: Raleigh mid-cycle meet-up at RedHat office
May 2014: Initial demo of Solum at Atlanta summit
June 2015: Solum accepted into OpenStack big tent

6. Community involvement
Companies that have contributed patches:
Rackspace, Independent, Numergy, Redhat, Mirantis, HP, AT&T, Oracle,
IBM, DreamHost
http://bit.ly/1PPhZ7I
http://bit.ly/1P3fBbW

7. Project details
• Wiki:
https://wiki.openstack.org/wiki/Solum
• Meetings:
Tuesdays 1700 UTC in #openstack-meeting-3
• IRC:
Channel: #solum on freenode
• Code:
https://github.com/openstack/solum
https://github.com/openstack/python-solumclient

8. Solum – What, why, how

9. What is Solum?
System which provides a declarative model for application
developers to deploy and run their application’s on OpenStack
starting from application’s source code
Why should you care?
– For operators, Solum provides ability to make your OpenStack cloud
more useful for your clouds’ application developers by simplifying the
process of deploying applications to it
– For developers, Solum provides an easy-to-use platform for building,
testing, and deploying applications on OpenStack clouds

10. Project goals
Application Stack Flexibility
Ability to support applications written in different languages and using different
application frameworks
Developer Productivity
Ability to perform CI/CD, integration with github
Add-On Services Extensibility
Ability to support different add-on services
Application Portability
Ability to deploy applications across different OpenStack clouds (use native
OpenStack services)

11. Solum abstractions
• App
• Languagepack (LP)
• Deployment Unit (DU)
• Workflow
• Add-on

12. Solum abstractions - App
Declarative application
definition
• source code repository
• run command
• port
• languagepack
version: 1
name: cherrypy
description: python web app
languagepack: python
source:
repository: https://github.com/
rackspace-solum-samples/solum-
python-sample-app.git
revision: master
workflow_config:
run_cmd: python app.py
ports:
- 80

13. Solum abstractions - Languagepack (LP)
• Docker image which contains application specific build and
runtime libraries
• A LP needs to implement Solum’s languagepack contract
A well-known file available at a known location on the languagepack image
/solum/bin/build.sh
• How to build a LP?
Dockerfile
• Operator-defined or user-defined

14. Solum abstractions - Languagepack Example: Python
FROM ubuntu:precise
MAINTAINER Murali Allada
<murali.allada@rackspace.com>
RUN apt-get -yqq update
RUN apt-get -yqq install python-pip
RUN apt-get -yqq install python-dev
COPY build.sh /solum/bin/
https://github.com/rackspace-solum-samples/solum-languagepack-python
#!/bin/bash
# Check if pip is installed
pip help
[[ $? != 0 ]] && echo python-pip is
not installed. && exit 1
# Install app dependencies
cd /app
pip install -r requirements.txt
build.shDockerfile

15. Solum abstractions - Deployment Unit (DU) (1/2)
• Docker image that is formed from the languagepack image with
application’s source code added to it
DU = LP + application source code
• Solum guarantees the DU contract
Application source code available at a known location on the DU image
/app
• How to build a DU?
– Construct Dockerfile with languagepack as the base image, application
source code injected, run command as the entry point
– Build the DU image from this Dockerfile

16. Solum abstractions - Deployment Unit (DU) (2/2)
• Building a DU
– Start from the specified languagepack LP
– Specify execution of languagepack’s ‘build.sh’
– Inject application source code
– Use the run command specified in app definition as the default
Entrypoint to run the DU
• LP and DU storage
Glance, Swift, Docker registry
16

17. Solum abstractions - Workflow
• Abstraction to represent execution of application deployment
consisting of one or more workflow stages
• Supported workflow stages
– Build DU (and store it for future use)
– Run unit tests, build DU
– Run unit tests, build DU, deploy DU (if unit tests pass)
– Build DU, deploy DU
– Deploy a previously built DU (not yet available)
• A workflow can be triggered from github webhooks

18. Solum abstractions - Add-ons
• Services needed by an application
E.g.: relational database such as Trove
• DU parameters
Solum supports ability to pass service’s connection parameters to
application DU

19. Architecture and system
details

20. High-level Architecture
Solum
API
Solum
Worker
Solum
DeployerQueue Queue
Queue
Solum
Conductor
Heat
Solum Database
Glance, Swift, Docker Registry
Builds LP and DU Deploys DU
- Decoupled architecture
- Asynchronous
communication

21. Demo
https://vimeo.com/143425822

22. Experience

23. Usage experience
• Languagepacks
- Internal test users mostly tended to use operator defined languagepacks
- Languagepacks for Wordpress and NodeJS+MongoDB application also
created by some
• Apps
Need for ease of application registration led to supporting interactive prompts
for getting app information
• Github integration
– Started with support for public repositories
– Internal use-cases required adding support for
• Private repositories
• CLI support for registering deploy keys
• CLI support for two-factor authentication

24. Challenges in building LPs and DUs
How to provide isolated environment for building LP and DU
Docker images and running untrusted unit tests on solum-
worker?
• Timeout mechanism to constrain and limit the running time of unit testing
scripts
• Isolated ‘git clone’ with resource constraints on CPU, memory, disk on
containers running unit testing scripts
• Running unit testing scripts as unprivileged user inside a container
• Easy-to-use CLI for operator to kill long running (malicious) containers

25. Reliability improvements
• Success rate of building and saving DU and LP images improved
from initial 80% to 98.8%
• DU and LP images saved in Glance and Swift using ‘docker save’
– Retry mechanism for performing git actions (git clone) and Docker actions
(build, save, load)
– Better use of Swift client to upload/download
– Race condition handling in ‘docker load’ and ‘docker rmi’
– Perform ‘docker rmi’ only for DU images on the worker node and not for LP images

26. DU Deployment - Supported options
• nova-docker driver
Works with DU images stored in Glance
• Heat + CoreOS VM + DU location provided through user_data
section
– Works with DU images stored in Swift and Docker registry
– Currently 1-1 mapping of DU to VM

27. Related Projects
• Deis
• CloudFoundry
• OpenShift
• Main value proposition of Solum is it is natively designed and
built for OpenStack
– Multi-tenancy through Keystone
– Docker image storage through Glance and Swift (also Docker registry)
– Deployment through Heat and Nova

28. Upcoming features

29. Key Features for Mitaka
• Application scaling
DU scheduling is the key
Leverage Heat + Magnum
• Application monitoring
DU monitoring is the key
Leverage Heat + Ceilometer
• Application Environments (dev/test/staging)
Ability to deploy pre-built DUs
• Multi-tier applications

30. Conclusions
• Solum is ready for early adopters
Appealing operators to consider deploying Solum in their OpenStack
clouds and provide feedback
• Solum is looking for application developers to try it
– Deploy applications starting from source code
– Build custom languagepacks
– Customize application workflows for continuous integration testing
– Integrate with Github (public and private)
– Easy-to-use CLI
• Exciting features have been planned for Mitaka
Come and help build J

31. Contact Information
Devdatta Kulkarni
devdatta.kulkarni@rackspace.com
(irc: devkulkarni)
Solum team members available on: #solum

32. O N E FA N AT I C A L P L AC E | S A N A N TO N I O , T X 7 8 21 8
U S S A L E S : 1 - 8 0 0 - 9 61 - 2 8 8 8 | U S S U P P O R T: 1 - 8 0 0 - 9 61 - 4 4 5 4 | W W W. R AC K S PAC E . C O M
© RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM
Thank you

33. FROM ubuntu:14.04
RUN apt-get -yqq update &&
apt-get -yqq install openjdk-7-jdk
RUN apt-get -yqq install maven
ENV JAVA_HOME /usr/lib/jvm/java-7-
openjdk-amd64
COPY bin/build.sh /solum/bin
https://github.com/rackspace-solum-samples/solum-languagepack-java
mvn -v
[[ $? != 0 ]] && echo "maven is not
installed" && exit 1
# build
cd /app
mvn -q package
FROM build.sh
Solum abstractions - Languagepack Example: Java

34. https://github.com/rackspace-solum-samples/solum-languagepack-wordpress
Dockerfile
--------------
Install PHP, required Apache modules
build.sh
------------
Empty
App run command
--------------------------
Setting the environment variables
and starting the app
Solum abstractions - Languagepack Example: Wordpress

35. Basic operational flow
• API receives a request to build and deploy an app
• API sends the request to the Worker
• Worker downloads the specified LP from configured storage
• Worker builds the DU and stores it
• Worker informs the Deployer to deploy the DU
• Deployer deploys the DU by calling Heat