Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Solum - OpenStack PaaS / ALM - Austin OpenStack summit

590 Aufrufe

Veröffentlicht am

Solum is a native Platform-as-a-service for OpenStack providing capabilities for application lifecycle management (testing, building, deploying, scaling).

Veröffentlicht in: Software
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Solum - OpenStack PaaS / ALM - Austin OpenStack summit

  1. 1. Solum PaaS/ALM for OpenStack Devdatta Kulkarni, PTL, Solum devdatta.kulkarni@rackspace.com (irc: devkulkarni)
  2. 2. Outline • Project goals • Early trials and use-cases • Solum architecture • Factors influencing adoption • Upcoming features • Question and answers 2
  3. 3. OpenStack User survey – April 2016 3
  4. 4. What is Solum? System which provides a declarative model for application developers to deploy and run their application’s on OpenStack starting from application’s source code Why should you care? – For developers, Solum provides an easy-to-use platform for building, testing, and deploying applications on OpenStack clouds – For operators, Solum provides ability to make their OpenStack cloud more useful to their application developers by enabling CI/CD capabilities for applications to OpenStack clouds 4
  5. 5. Project goals Developer Productivity Ability to perform CI/CD, integration with github, integration with existing CI solutions Application Stack Flexibility Ability to support applications written in different languages Add-On Services Extensibility Ability to support different add-on services Application Portability Ability to deploy applications across different OpenStack clouds 5
  6. 6. Solum - System for building and deploying applications to OpenStack Clouds Solum in context with other OpenStack services Cinder (Block) Manila (File) Trove (DB) Neutron (Network) Nova (VM) Glance (Image) Magnum (Container orchestration & lifecycle) Heat (Orchestration) Solum (App lifecycle) Keystone (Identity) Horizon (UI) Murano (App catalog) Mistral (Workflow) Ironic (Baremetal) Swift (Object) Barbican (KeyMgmt) Ceilometer (Monitoring) 6
  7. 7. Early trials & usecases 7
  8. 8. Solum in the wild 8
  9. 9. Solum in use Users/use-cases: • Wipro • Rackspace • Paderborn University Interest / References: 9 https://blog.jasoncallaway.com/ http://www.tcpcloud.eu/en/blog/2015/06/27/beyond-horizon-part-1/ http://ronaldbradford.com/blog/using-your-devstack-cloud-2016-04-05/ http://egonzalez.org/nova-docker-driver/ https://www.ctl.io/developers/blog/post/openstack-solum-on-the-centurylink- cloud/
  10. 10. How to use existing Jenkins setup with Solum for performing seamless application CI/CD on OpenStack? Solum features used: App image storage in Glance, API to deploy a pre-built DU Usecase 1 Wipro: Jenkins for CI, Solum for CD Build Test Release Deploy Continuous Integration with Jenkins Continuous Deployment with Solum 10
  11. 11. Easily deploy applications to Carina (container cluster service from Rackspace) Usecase 2 Rackspace: Deploying apps to Docker Swarm User sends command as “Deploy Application” from Rackspace UI Solum deploys requested application on Carina Rackspace UI shows the deployment progress status Solum features used: Pluggable deployer architecture; Built a Carina specific deployer in Solum 11
  12. 12. Easily deploy applications to Virtual Machines Usecase 3 Rackspace: Deploying apps to VMs Solum features used: Pluggable deployer architecture, Used Solum’s Heat deployer User sends command as “Deploy Application” from Rackspace UI Solum deploys requested application on Nova using Heat Rackspace UI shows the deployment progress status 12
  13. 13. Continuous integration and testing of Chef recipes Solum features used: Custom languagepacks: Developed a custom languagepack containing chef testing tools (rubocop, foodcritic, chef-spec, test-kitchen) https://github.com/openstack/solum/tree/master/examples/language-packs/chef Integrate with Private github repositories Usecase 4 Rackspace: Testing Chef recipes User commits Chef code to github repo Solum runs Chef tests – test- kitchen, rubocop, chef- spec, foodcritic Solum saves test results in swift and sends notification to github 13
  14. 14. Considering to build and deploy services from source code with an existing Heat template Useful Solum features: Build application DUs,; integration with public/private git repositories Usecase 5 Paderborn Univ: Deploying multiple services User defines Heat HOT Custom Heat plugin calls Solum to build images Heat deploys the ensemble 14
  15. 15. Solum architecture 15
  16. 16. Solum abstractions and concepts • Languagepack (LP) • App • Deployment Unit (DU) • Workflow • Parameters 16
  17. 17. Solum abstraction - Languagepack (LP) • A Docker image that contains libraries, packages, and tools required for testing, building, compiling an application – Examples: • Python applications need: pip, tox, python-mysqldb • Java applications need: jdk, jre, maven • Chef recipes: chefdk, rubocop, test-kitchen, foodcritic • Application-specific languagepacks – Different languagepacks for different life-cycle stages of an application • Testing libraries may be different than libraries for building application artifacts – Multi-language applications • Combine all libraries and tools from different languages in a single languagepack • Operator-defined languagepacks, user-defined languagepacks $ solum languagepack create python https://github.com/rackspace-solum-samples/solum-languagepack-python.git 17
  18. 18. Solum abstraction - Languagepack Example: Python FROM ubuntu:precise MAINTAINER Murali Allada <murali.allada@rackspace.com> RUN apt-get -yqq update RUN apt-get -yqq install python-pip RUN apt-get -yqq install python-dev COPY build.sh /solum/bin/ https://github.com/rackspace-solum-samples/solum-languagepack-python #!/bin/bash # Check if pip is installed pip help [[ $? != 0 ]] && echo python-pip is not installed. && exit 1 # Install app dependencies cd /app pip install -r requirements.txt 18
  19. 19. Solum – sample languagepacks https://github.com/rackspace-solum-samples/solum-languagepack-python https://github.com/rackspace-solum-samples/solum-languagepack-java https://github.com/rackspace-solum-samples/solum-languagepack-nodejs https://github.com/rackspace-solum-samples/solum-languagepack- nodejswithmongo https://github.com/rackspace-solum-samples/solum-languagepack-wordpress https://github.com/openstack/solum/tree/master/examples/language-packs/chef 19
  20. 20. Other languagepacks to support 20
  21. 21. Solum abstraction - App Declarative application definition • source code repository • run command • port • languagepack version: 1 name: cherrypy description: python web app languagepack: python source: repository: https://github.com/rackspace-solum- samples/solum-python-sample- app.git revision: master workflow_config: run_cmd: python app.py ports: - 80 $ solum app register --app-file appfile.yaml Reason to specify languagepack: • App developer knows what libraries are required by their app 21
  22. 22. Solum concept - Deployment Unit (DU) • Docker image that is formed from the languagepack image with application’s source code added to it DU = LP + application source code • Solum guarantees the DU contract Application source code available at a known location on the DU image /app • How to build a DU? – Construct Dockerfile with languagepack as the base image, application source code injected, run command as the entry point – Build the DU image from this Dockerfile • LP and DU storage Glance and Swift 22
  23. 23. DU Deployment: Supported options • Nova-docker driver Works with DU images stored in Glance • Heat + VM image + DU location provided through user_data section Works with DU images stored in Swift/Docker registry 1-1 mapping of DU container to VM • Carina deployer in Rackspace’s Solum environment Works with Rackspace Carina 23
  24. 24. Solum abstraction - Workflow • Abstraction to represent execution of application deployment action • Supported application deployment actions (workflows) –Build DU –Run unit tests and build DU –Run unit tests, build DU, and deploy DU –Build DU and deploy DU –Deploy a pre-built DU • Workflows can be triggered from github webhooks $ solum app deploy cherrypy 24
  25. 25. Solum concept – Parameters • Application parameters – Services needed by an application • E.g.: Connection parameters for relational database such as Trove – These are injected into the DU • Environmental parameters – Parameters intended for Solum to use • E.g.: Carina cluster credentials • Parameters specified at: – application registration time – application deploy time (upcoming feature) $ solum app create --app-file appfile.yaml --param-file params.yaml 25
  26. 26. Solum – sample apps https://github.com/openstack/solum/blob/master/examples/apps/python_app.yaml https://github.com/openstack/solum/blob/master/examples/apps/java_app.yaml https://github.com/rackspace-solum-samples/solum-wordpress-sample-app https://github.com/openstack/solum/blob/master/examples/apps/nodejs_app.yaml 26
  27. 27. Summary of current Solum features • Create languagepacks • Register, deploy, scale an app • Update an app keeping the application URL same • Specify application parameters when registering an app • Integration with public and private github repositories (via github webooks) • Deploy app using different compute form factors – nova-docker – VMs – Container cluster (such as Docker Swarm) • Environments (dev/test/staging/production) – Possible by creating different app for each env and deploying same DU for each 27
  28. 28. High-level Architecture Solum API Solum Worker Solum Deployer Queue Queue Queue Solum Conductor Heat Database Glance/Swift Builds LP; run unit tests; Build DU Deploys DU - Decoupled architecture - Asynchronous communication 28
  29. 29. Factors influencing adoption 29
  30. 30. Usage experience • Languagepacks - Languagepacks built: python, java, php, nodejs, angularjs, chef testing tools - Internal test users at Rackspace tended to use operator defined languagepacks - Need to install appropriate OS packages in addition to the language libraries/tools - python-msqldb not sufficient, needed OS-level mysql package - Languagepack creation is iterative process - Learning curve – how to lower this barrier is question? • Apps Need for ease of application registration led to supporting interactive prompts for getting app information • Github webhooks – Started with support for public repositories – Internal use-cases required adding support for • Private repositories • CLI support for registering deploy keys • CLI support for two-factor authentication 30
  31. 31. Building LPs and DUs How to provide isolated environment for building LP and DU Docker images and running untrusted unit test scripts? • Isolated ‘git clone’ with resource constraints on CPU, memory, disk on containers running unit testing scripts • Running unit testing scripts as unprivileged user inside a container • Timeout mechanism to constrain and limit the running time of unit testing scripts • Easy-to-use CLI for operator to kill long running (malicious) containers • On Rackspace version of Solum – using Carina for building LPs, running unit tests, and building DUs This provides a natural isolated environment 31
  32. 32. Reliability Experience • DU and LP images saved in Glance and Swift using ‘docker save’ • Success rate of building and saving archives improved from 80% to 98.8% –Retry mechanism for performing git actions (git clone) and Docker actions (build, save, load) –Better use of Swift client to upload/download archives –Race condition handling in ‘docker load’ and ‘docker rmi’ Perform ‘docker rmi’ only for DU images on the worker node and not for LP images Small risk of Worker running out of disk space 32
  33. 33. Performance Experimental setup - 2 servers, 10 workers, 10 deployers, Deployer: Heat with deployment to VMs Applications used - Mix of Python and Java applications Deployed 330 applications in 30 minutesResults 330 Queued • 3 in ERROR_STACK_CREATE_FAILED • 327 in READY (99.09%) Average time from QUEUED to READY = 348 seconds (6.5 minutes) • 36.8 seconds for unit testing • 99 seconds to build DU • 41 seconds to deploy DU via Heat • 120 seconds to start the web app (wget, docker load, docker run) • Other time spent in queuing. We are seeing faster times when using Carina deployer (within Rackspace Solum setup) 33
  34. 34. Community involvement http://stackalytics.com/?release=all&project_type=openstack&module=solum-group 34
  35. 35. Upcoming features 35
  36. 36. Key Features for Newton • Multi-tier/micro-service applications – Currently possible using the parameter feature – We plan to build native support in Solum during Newton (app file format changes) • Infrastructure choice for deploying applications – Operator vs. App developer choice • VMs for isolation • Bare metal for performance • Container clusters for packaging • Application monitoring DU monitoring is the key Leverage Heat + Ceilometer 36
  37. 37. Other Features Health monitoring of Solum services and apps Documentation, examples 37 Solum Features/Improvements
  38. 38. Conclusions • Good news Solum is getting interest from users. • Rackspace • Wipro • Paderborn University • Current capabilities •CI/CD of applications starting from app source code natively on OpenStack •Building custom languagepacks •Integration with Github (public and private) •CLI • Exciting features have been planned for Newton 38
  39. 39. References Wiki: https://wiki.openstack.org/wiki/Solum Setting up development environment: https://wiki.openstack.org/wiki/Solum/solum-development-setup Meetings: Tuesdays 1700 UTC in #openstack-meeting-3 IRC: Channel: #solum on freenode Code: https://github.com/openstack/solum https://github.com/openstack/python-solumclient Demo: https://vimeo.com/163874891 https://vimeo.com/143425822 39
  40. 40. O N E FA N AT I C A L P L A C E | S A N A N T O N I O , T X 7 8 2 1 8 U S S A L E S : 1 - 8 0 0 - 9 61 - 2 8 8 8 | U S S U P P O R T: 1 - 8 0 0 - 9 61 - 4 4 5 4 | W W W. R AC K S PAC E . C O M © RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED S TATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM Thank you
  41. 41. FROM ubuntu:14.04 RUN apt-get -yqq update && apt-get -yqq install openjdk-7-jdk RUN apt-get -yqq install maven ENV JAVA_HOME /usr/lib/jvm/java-7- openjdk-amd64 COPY bin/build.sh /solum/bin https://github.com/rackspace-solum-samples/solum-languagepack-java mvn -v [[ $? != 0 ]] && echo "maven is not installed" && exit 1 # build cd /app mvn -q package Dockerfile build.sh Solum abstractions - Languagepack Example: Java 41
  42. 42. https://github.com/rackspace-solum-samples/solum-languagepack-wordpress Dockerfile -------------- Install PHP, required Apache modules build.sh ------------ Empty App run command -------------------------- Setting the environment variables and starting the app Solum abstractions - Languagepack Example: Wordpress 42
  43. 43. Basic operational flow • API receives a request to build and deploy an app • API sends the request to the Worker • Worker downloads the specified LP from configured storage (Swift or Glance) • Worker builds the DU and stores it • Worker informs the Deployer to deploy the DU • Deployer deploys the DU by calling Heat 43
  44. 44. Related Projects • Deis • CloudFoundry • OpenShift 44
  45. 45. Performance Experimental setup (2 servers, 10 workers, 10 deployers) – MemTotal: 8171504 kB – Architecture: x86_64 – CPU(s): 8 – Vendor ID: GenuineIntel – CPU MHz: 2593.604 – Hypervisor vendor: Xen – L1d cache: 32K – L1i cache: 32K – L2 cache: 256K – L3 cache: 20480K 45
  46. 46. OpenStack User survey – April 2016 46

×