CTO Chris Swan's talk from Open Networking User Group (ONUG) New York conference presentation "Container Networks and Network Containment"
Containers aren’t a new thing, but the Docker project has made them a hot topic as organisations look at new ways to build, ship and run their applications. This brings new challenges for the network as containers are likely to be ten times as numerous as virtual machines. At the same time there is regulatory pressure to move away from the flat LAN model and deliver greater separation and segregation. This presentation will look at how these two forces are coming together, firstly by examining how containers are networked and some of the new approaches and challenges that come with that. This will be followed by a look at how overlay networks are being deployed to achieve ‘microsegmentation’, and ultimately drive a shift towards application centric networking. Of course these forces will collide, bringing us to contained networks of containers.
ONUG 2014: Container Networks and Network Containment
1. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
October 28-29, 2014
2. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Container Networks and
Network Containment
Chris Swan
CTO CohesiveFT
@cpswan
3. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Part 1 – Container Networking
3
4. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
4
TL;DR
docker0 bridge is the heart of default networking
Plus some iptables magic
Docker can help link your containers (on a single host)
But it’s easier with a compositing tool
There are advanced options
On a single host
On multi hosts
and advanced tools
5. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Do I first need to explain
Docker and containers?
5
6. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Build, Ship > Run?
6
Image credit http://www.mediaagility.com/2014/docker-the-next-big-thing-on-cloud/
7. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Docker Hub
7
Image credit http://blog.docker.com/2014/06/announcing-docker-hub-and-official-repositories/
8. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Demo time
9. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Why me?
9
10. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Conceived last summer – released this April
1
11. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
The basics
1
12. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
1
Let’s start with a regular host
eth0
10.0.1.1
13. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
1
Install Docker
eth0
10.0.1.1
docker0
172.17.42.1
14. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
1
Start a container
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
15. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
1
Start another container
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d
16. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
iptables magic
1
17. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
1
Connecting to the outside world
$ sudo iptables -t nat -L –n
...
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
MASQUERADE all -- 172.17.0.0/16 !172.17.0.0/16
...
18. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
1
Connecting from the outside world
$ sudo docker run –dp 1880:1880 cpswan/node-red
$ sudo docker ps
CONTAINER ID IMAGE COMMAND
CREATED STATUS PORTS
NAMES
7696169d9438 cpswan/node-red:latest node red.js 2
weeks ago Up 2 weeks 0.0.0.0:1880->1880/tcp
backstabbing_davinci
$ sudo iptables -t nat -L –n
...
Chain DOCKER (2 references)
target prot opt source destination
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp
dpt:1880 to:172.17.0.7:1880
19. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Container linking
1
20. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
2
From the docker command line
From the outside:
# start the database
sudo docker run -dp 3306:3306 --name todomvcdb
-v /data/mysql:/var/lib/mysql cpswan/todomvc.mysql
# start the app server
sudo docker run -dp 4567:4567 --name todomvcapp
--link todomvcdb:db cpswan/todomvc.sinatra
On the inside:
dburl = 'mysql://root:pa55Word@' + ENV['DB_PORT_3306_TCP_ADDR'] +
'/todomvc'
DataMapper.setup(:default, dburl)
21. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
2
Simplify life with Fig
fig.yml:
todomvcdb:
image: cpswan/todomvc.mysql
expose:
- "3306"
volumes:
- /data/mysql:/var/lib/mysql
todomvcapp:
image: cpswan/todomvc.sinatra
ports:
- "4567:4567"
links:
- todomvcdb:db
I still need this on the inside:
dburl = 'mysql://root:pa55Word@' + ENV['DB_PORT_3306_TCP_ADDR'] +
'/todomvc'
DataMapper.setup(:default, dburl)
22. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Other networking modes
2
23. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
2
--net=host
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d
24. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
2
--net=container:$container2
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d
25. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
2
--net=none
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d
26. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Connecting containers between
machines
2
27. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
2
Marek Goldmann did this with OVS
28. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
2
A more generic approach (ODCA)
29. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Flocker
2
30. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Weave
3
31. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Still want more…
3
32. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
3
Pipework etc.
Pipework:
• Create bridges
• Attach to container interfaces
• Attach to host interfaces
• and much more…
Tenus:
• Golang package offering programmatic
network configuration along similar lines to
Pipework
33. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
libchan
‘A low level component that we can use as a communication layer
that we can use across the board for all the different aspects of
communication within Docker’
Solomon Hykes – DockerCon 2014 (my emphasis)
What it is – Golang like channels over the network
‘A lightweight communication protocol for distributed systems’
What it does – yet to be revealed
3
34. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Gotchas
3
35. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
3
Our old enemy the network hub
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d
36. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
3
A bit like a home network
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d
37. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Host as router can be painful
• VirtualBox requires specific network adaptors (in a specific configuration) to
play nicely with pipework
• Even with source/destination checks disabled pipework won’t play nicely on
EC2
– Mileage may vary on other clouds, but some don’t even have the option to flick that
bit (or make it very hard to get at)
3
38. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
The end of this part (nearly)
3
39. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Docker makes a great place to run
L4-7 Network Application Services
3
40. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
4
TL;DR
docker0 bridge is the heart of default networking
Plus some iptables magic
Docker can help link your containers (on a single host)
But it’s easier with a compositing tool
There are advanced options
On a single host
On multi hosts
and advanced tools
41. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Part 2 –Network Containment
4
42. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
4
TL;DR
Hard shell and soft centre has never served us well
The pressure to move on is mounting
Finer grained network segregation was too expensive in
hardware
Software makes it achievable
We’re seeing the dawn of application centric networking
and the Application Security Controller
43. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Enterprise networks and perimeters
4
44. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
The confectionary networking model
Hard crunchy perimeter
Soft chewy centre
Image credit CC by Sandra Fauconnier
https://www.flickr.com/photos/spinster/4369608/
45. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Pretty much everybody has a ‘demilitarized zone’
DMZ
Intranet
46. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Sophisticated organisations have an application server zone
DMZ
Intranet
ASZ
47. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Global scale makes things messy
DMZ
Intranet
ASZ
DMZ
ASZ
DMZ
ASZ
Europe Americas Asia
48. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Some even have a ‘domain zoning concept’
49. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
This is VERY expensive when done with hardware
50. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
But potentially cheap and flexible if done in software
51. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
‘Microsegmentation’ – the VMware view
Image credit http://vinfrastructure.it/2014/09/micro-segmentation-with-nsx/
52. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
What’s driving this?
53. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Are you being asked to look at this?
54. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
In particular this:
55. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Application centric networking
56. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
What’s the right granularity?
Microservice Service Service family
57. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
The sweet spot likely depends on containment of business data
Microservice Service Service family
58. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
To each their own
Encrypted
overlay
Firewall
NIDS TLS
Cache
Load balancer
Proxy
59. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Using an ‘Application Security Controller’
Encrypted
overlay
Firewall
NIDS TLS
Cache
Load balancer
Proxy
60. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Wrapping up
61. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
6
TL;DR
Hard shell and soft centre has never served us well
The pressure to move on is mounting
Finer grained network segregation was too expensive in
hardware
Software makes it achievable
We’re seeing the dawn of application centric networking
and the Application Security Controller
62. Copyright 2014 Open Networking User Group. All Rights Reserved Confidential Not For Distribution
Questions?
chris.swan@cohesiveft.com
@cpswan