StorageOS' Karolis Rusenas discusses why storage for containers shouldn't be annoying at Container Camp UK.

1. Storage for containers
shouldn’t be annoying
Karolis Rusenas
@KRusenas
https://github.com/rusenask
karolis.rusenas@storageos.com

2. Agile Storage. On-Demand, Anywhere, Made Easy.
What we care about
 Developer experience
 Use existing tools (create and use Docker volumes without ever interacting with StorageOS
directly (UI/CLI/API).
 Reasonable defaults - compression, encryption, replication…
 Operating experience
 Run StorageOS as a container, separate image or whatever way you like, it’s just a binary! :)
 API first - UI and CLI reuses same single HTTP API, easy to automate stuff.
 Pluggable configuration backends for whatever you have in your stack - Consul, Zookeeper,
BoltDB, Etcd (easy to add custom ones).
 Security
 Perfomance
2

3. Challenges
• Persistent storage
• Networking
• Security
• Infrastructure
• Orchestration

4. Our targets
• Persistent storage
• Networking
• Security
• Infrastructure
• Orchestration

5. Persistent storage

6. Jim was almost ready to implement his first app
with persistent storage running in Docker container

7. How easy is it to get persistent
storage with StorageOS?

9. That’s the line
• sudo docker run -d --name test-redis01 -v test-dev-redis01:/data --volume-driver=storageos
redis redis-server --appendonly yes
Usual stuff
Let’s give our new friend a name
Telling Docker to use
StorageOS driver
More of the usual stuff

11. Infrastructure

12. Infrastructure
• Cloud providers such as GCE or AWS provide EBS
volumes for persistent storage.

13. You can always build
something on your own…

14. On-premises infrastructure
& StorageOS
• Benefits of containers without moving to cloud by providing EBS volume alternative via
Docker volume plugin.
• Hyper-converged mode - run your workloads on the same nodes as StorageOS
controllers for maximum performance.
• Client mode - present virtual volumes to Docker containers, easy access to remote
volumes.
• QoS
• Compression
• Data deduplication
• E2E encryption

15. Agile Storage. On-Demand, Anywhere, Made Easy.
StorageOS Use Cases
15
 Stateful Containers for Databases and Fast DB
recovery
 Continuous Integration/Delivery
 Secure Cloud Mobility and Cost Reduction  Performance Acceleration and
Volume Management
API

16. StorageOS insides

17. 17
Controller Node
control plane
HTTP
API
Nats
UI
(Angular)
Docker
plugin
Scheduler
Dataplane
drivers
Supervisor 5 secret binaries
Configfs
data plane

18. 18
Controller Node
control plane
HTTP
API
Nats
UI
(Angular)
Docker
plugin
Scheduler
Dataplane
drivers
Supervisor 5 secret binaries
Configfs
Client Node
control plane
Docker
plugin
Nats
HA
Client
Supervisor
5 secret
binaries
data plane

19. 19
controplane
High level overview
controplane controplane
dataplane dataplane dataplane
nats
Consul Consul Consul
http
client
http/nats
Docker dataplane
server

20. Our stack
• Consul/BoltDB - store configuration. BoltDB is useful
when running a single node or during
development/testing.
• Nats - messaging system
• Go - control plane is written in this awesome
language
• C - data plane, mostly for speed and available
libraries

21. Consul/BoltDB
 KV store, easy to use, backup.
 Service discovery (when using
BoltDB it’s not important since we
assume that you are running a single
node)
 Leadership election - some
components of the system should be
running only on one node so they
are all fighting for leadership (i.e.
scheduler, retry logic).
 Split brain detection
21

22. Leadership election.. why?
 Sometimes (and quite often) you need only one
node in your distributed system performing specific
actions, i.e. scheduling, retrying some actions..
 Most of the distributed KV stores implement locking
mechanism on keys, that could be used to elect
leaders and detect leader failures.
 Check out https://github.com/docker/leadership -
probably not enough code there to include it as a
library though.
22

23. 23
Node 1 Node 2 Node 3 Node 4 Node 5
Leader’s
key
All nodes try to
acquire a lock on
specific key

24. 24
Node 1
(leader)
Node 2 Node 3 Node 4 Node 5
Leader’s
key
Only one will
succeed
P.S. Don’t forget to use locks with TTL!

25. Nats (https://nats.io)
 Lightweight
 Server is just a goroutine in your
main process
 Instant messaging between
components
 Simple pub/sub or request/reply
syntax
 Mesh networking
25

26. Nats (https://nats.io) auto discovery:
https://github.com/nats-io/gnatsd/issues/126
26

27. Safer messaging
27
Messaging
wrapper
Dispatch
event
Persist
event in KV
store
Publish
event to
Nats
Send event:
Receive event:
Receive
event
Update
event to
“receive”
state
Do stuff
Then we can have a
process that
constantly looks for
forgotten events :)

28. Statik (https://github.com/rakyll/statik)
 Compile your static assets into Go
binary!
28

29. Suture (https://github.com/thejerf/suture)
 Erlang-ish supervisor trees for Go
 Stable
 Combined with statik and go-bindata
you can drastically simplify your
deployments
29

30. Using Docker’s healthcheck functionality
(added in 1.12)
 https://docs.docker.com/engine/reference/builder/#/healthcheck
 Dockerfile:
 $docker ps
30
Useful when you have
several dependencies,
like KV store.

31. Thanks!
@Storage_OS
We are hiring!
http://storageos.com/careers/