Shows how to abstract Cloud specific infrastructure in AWS, Azure and own DC using a demo application written in Vertx.

1. Sascha Möllering | zanox AG
Docker in the Cloud

2. About me
Sascha Möllering
sascha@autoscaling.io
sascha.moellering@zanox.com
Lead Engineer / DevOps Hipster
zanox AG
http://autoscaling.io
@sascha242

3. About me
Java Magazin 11.14
Architektur im Kontext der Cloud: Patterns und Best Prac
AWS Virtual Cloud Summit
Continuous delivery in AWS using Docker
http://aws-summit.de/
Entwickler Spezial: Docker
Über den Wolken: Ein Erfahrungsbericht zum Docker-Ein

4. Agenda
• Problem?
• Real World example
• Docker implementations
• Deployment
• Q&A

6. Problem?
• Using different
Cloud providers
• Additionally: own DC
• Different
infrastructure
• Different
deployment
mechanisms

7. Solution!
• Different applications!
• For each deployment target -> different
package!

10. • One container to rule them all!
• Application detects deployment target
• Application uses specific infrastructure
Real Solution!

11. Real World example
• JVM based event-driven application
• Ingests data into bus system
• Uses Redis as a cache
• Base framework is Vert.x
• Pre-build Docker image

12. Real World example

13. Real World example
• Deployment targets:
– Own data center
– Amazon Web Services
– Microsoft Azure

14. Real World example
• Our own data center:
– Linux servers with Docker 1.6
– Apache Kafka 0.8.1/0.8.2
• Topics and partitions
• Easy to write, hard to read from
– Redis 2.8

15. Real World example
• Amazon Web Services:
– EC2 instance with Docker 1.6
– Amazon Kinesis
• Amazon’s implementation of Kafka (?)
• Streams and shards
• KCL does the heavy lifting
– ElastiCache with Redis engine
• Amazon’s implementation of Redis

16. Real World example
• Microsoft Azure:
– Azure Ubuntu VM with Docker 1.6
– Azure Service Bus
• Bus system in Azure
• AMQP and JMS
• Problem: no partitioning for Java
– Azure Cache
• Redis implementation in Azure

17. Real World example
• For Redis: Jedis client in all cases
• Difference: bus system
– mod-kafka
– mod-kinesis
– mod-service-bus

18. Real World example
• mod-kafka:
– Vert.x module for Apache Kafka
– Recieves events by other verticles
– Sends events to Kafka broker
– Either Strings or binary data
– Supports StatsD
– Based on Apache Kafka Java API

19. Real World example
• mod-kinesis:
– Vert.x module for Amazon Kinesis
– Recieves events by other verticles
– Sends events to Kinesis stream
– Either Strings or binary data
– Support for partitioning in events
– Based on AWS Java SDK

20. Real World example
• mod-service-bus:
– Vert.x module for Azure Service Bus
– Recieves events by other verticles
– Support for binary data only ATM
– No partitioning (Service Bus problem)
– Based on AMQP and JMS

21. Real World example
But … how can we
detect where the
application is running?

22. Real World example
http://169.254.169.254/latest/meta-data

23. Real World example
boolean isAzure = true;
try {
RoleEnvironment.getRoles();
} catch (Exception exc) {
isAzure = false;}

24. Real World example
• Problem: not working in Docker container
• Alternative: check for /etc/waagent.conf
• File has to be mapped in Docker container

25. Real World example
• StarterVerticle pattern:
– Bootstrapping Vert.x application
– Start all verticles in StarterVerticle
– Deploy all modules in StartVerticle

26. Real World example

27. Docker implementations

28. Docker implementations
• Basic implementation:
– EC2 using User Data
– Install Docker and start service
– Either pull Container in User Data script
– Or Remote API

29. Docker implementations
• AWS OpsWorks:
– No “official” support
– Docker Chef Cookbook by AWS
– Problem: Docker Images are build in the
Cookbook

30. Docker implementations
• Amazon Elastic Beanstalk:
– Support for single/multi container
environments
– ELB support on container level
– Multi container environments -> ECS
resources
– Running containers locally is possible

31. Docker implementations
• Amazon EC2 Container Service:
– Container management service for Docker
– Description language for applications (“task
definition”)
– Supports batch mode (taks that only run
once)
– Supports long running taks
– ELB support on container level

32. Docker implementations

33. Docker implementations
• xplat-cli:
– Support for:
• Ubuntu from Azure Image Gallery
• Custom Linux image with Azure Linux VM
Agent
– Similar to plain EC2 approach

34. Docker implementations
• xplat-cli:
– azure vm docker create
– Docker remote API

36. Deployment

37. Deployment
Amazon Cloud
Workstation
GitHub Jenkins
Python/Boto
Docker Image
Docker Registry S3 bucket with
Docker Images

38. Deployment

39. Deployment

40. Deployment
• AWS OpsWorks:
– Docker Chef Cookbook by AWS
– Add a new stack containing the Cookbook
– Add a new layer and add custom recipes
– Docker Images are build in the Cookbook
– Use custom JSON to modify stack
configuration

41. Deployment
• Amazon Elastic Beanstalk:
– Create an application using console or CLI
– Create Dockerrun.aws.json to deploy a
Docker container
– Create a ZIP-file containing application
files (optional)

42. Deployment
1. Create a cluster
2. Create a task definition
3. Create a service
4. Configure cluster
5. Run task

43. Deployment

44. Deployment
• Remote API is exposed
docker --tls -H tcp://<vm-name-
you-used>.cloudapp.net:4243
info

45. Deployment
• Useful Docker clients:
– Spotify Docker Client
– docker-py

46. Deployment

47. Deployment

48. Q&A