This presentation was delivered at ContainerCon North America 2016 that was held in Toronto. This talk examines the history of Docker Swarm and libNetwork and Storage to see how the increased complexity in the container ecosystem is actually simplified over time.
8. 8 of Y
• Swarm 1.0
– Required external Key/Value for scale and use with libNetwork
– Container restarts required additional flags
• Kubernetes
– https://github.com/kelseyhightower/kubernetes-the-hard-way
• Mesos
– Bring your own framework with specific syntax
Orchestration wasn’t easy
9. 9 of Y
• Integration of SwarmKit
– https://github.com/docker/swarmkit
• Integrated into the Docker engine
• Desired state service declaration with reconciliation
• Decentralized service with managers and workers
• Multi-host networking with integrated service discovery and load balancing
• Rolling updates for services
• Secure by default using TLS mutual authentication and encrypted communication
Docker 1.12 with Swarm Mode
13. 13 of Y
What's the problem?
• When I run a persistent
application in a container, where
does my data get stored?
– The container holds the data
directory and structure of the
entire application
– Optionally use local volumes
• Stateless applications work well
– nginx, httpd, kibana, haproxy,
memcached, solr, celery
$ docker run -v redisData:/data redis
redisData
/etc /var
/bin /opt
/data
14. 14 of Y
What's the problem?
• Lose a container
– Lose the data
• Lose a server
– Lose the data
• Local data storage
– Failed hard drives or failed RAID
– Can not scale beyond the
physical limit of the server
/etc /var
/bin /opt
/data
16. 16 of Y
REX-Ray
github.com/emccode/rexray
• Integrations:
– Docker, Mesos, Marathon, Kubernetes
• Support for multiple storage platforms
– On-Prem:
› EMC ScaleIO, XtremIO, Isilon, VMAX
› Oracle VirtualBox
– Cloud:
› AWS
› GCE
› RackSpace/OpenStack Cinder
• Only solution to provide High Availability
• Open Source and in sync with Docker
development cycle
• Single binary with simple installation using
curl | sh and configured using 1 YAML file
17. 17 of Y
Solving the problem
• Docker created a volume plugin
framework, allowing volume
plugins to receive API requests
from the Docker daemon to
perform storage operations
– Create/Mount/Unmount/Delete/Sn
apshot
$ docker run --volume-driver=rexray -v
redisData:/data redis
/redisData
/etc /var
/bin /opt
/data
18. 18 of Y
Solving the problem
• Lose the container or lose the
server
– Data persists and remains in tact
on the remote storage platform
/etc /var
/bin /opt
/redisData
19. 19 of Y
Solving the problem
• Attach the volume to a new
container on a different host
– Equivalent of a hard reset.
Application starts and resumes
from the last write to disk
– Container schedulers can
perform automated failover
• Scalability
– Application data can scale to the
maximum supported by the
storage platform
/etc /var
/bin /opt
/data /redisData
20. 20 of Y
How it works
$ docker run
--volume-driver=rexray
-v redisData:/data
redis
redisData
/etc /var
/bin /opt
/data
Storage Platform
Persistent Volume
22. 22 of Y
• What we learned
– Clustering, networking, and failover with persistent applications in containers is
easier than ever before
• Get Hands On!
– emccode.com
– github.com/emccode/rexray
– github.com/emccode/vagrant
Kendrick Coleman
Developer Advocate
@kendrickcoleman
github.com/kacole2
23. 23 of Y
Who is {code}?
• Open Source initiative focused on next-
generation infrastructure
• Contribute to meaningful Open Source
Software (OSS) projects
• Create OSS integrations to keep EMC relevant
when container-based infrastructure enters a
conversation
• Driving awareness of OSS opportunities with
product teams
• Acting in the interest of building a community
Hinweis der Redaktion
Operated by a steering wheel and two pedals
so what types of applications run in containers.. we just showed you a simple proxy in the first demo. But we at EMC {code} view the container as the next unit of infrastructure. We believe you can easily transition from a Virtual Machine first policy to container first. so benefits of containers stretch to all types of applications
Here are the top 12 applications as reported by docker being downloaded from Docker Hub. 7 of these require a level of persistence to survive container failures or restarts. You have databases like Mongo and Postgres but also applications like Wordpress.
In this example, our application is storing data in “/mydata”
In this example, our application is storing data in “/mydata”
The first project EMC {code} created to integrate storage to Docker is Project REX-Ray.
There are many benefits of using REX-Ray in a container environment, and many of our customers are already using it for their container environments. We have also seen several other companies embrace and integrate REX-Ray into their commercial container management platforms.
In this example, our application is storing data in “/mydata”
First, a few things about the team that has made this possible.
The EMC {code} team is a team made up of open source software engineers and developer advocates, focused on making EMC a well-known name within the open source community.
We will focus on one of their projects, REX-Ray, in this presentation.