1.
DEPLOYING AND
SCALING
MICROSERVICES
Sam Newman
YOW! 2015

2.
@samnewman

3.
@samnewman
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS

4.
@samnewman
Core Principles

5.
@samnewman
Artifacts
Core Principles

6.
@samnewman
Artifacts
Core Principles
Platforms

7.
@samnewman
Artifacts
Core Principles
Platforms

8.
@samnewman
Accounts
Returns
Invoicing
Shipping
Inventory
Customer
Service

9.
@samnewman
Independent Deployability

10.
@samnewman
Accounts
Returns
v345
Invoicing
Inventory
Customer
Service
Shipping
v123

11.
@samnewman
Accounts
Invoicing
Shipping
v456
Inventory
Customer
Service
Returns
v890

12.
@samnewman
https://www.flickr.com/photos/torkildr/3462607995/

13.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/

14.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control

15.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control

16.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control

17.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control
!

18.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control
! !

19.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control
! ! !

20.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control
! ! !

21.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control
! ! !
One Artifact For All Environments

22.
@samnewman
TestsBuild Performance ProdUAT
https://www.flickr.com/
Source
Control
! ! !
One Artifact For All Environments
Same Deployment Process Everywhere

23.
@samnewman
$ deploy Returns v456 Production

24.
@samnewman
$ deploy Returns v456 Production
Service Name

25.
@samnewman
$ deploy Returns v456 Production
Service Name
Version

26.
@samnewman
$ deploy Returns v456 Production
local
Service Name
Version

27.
@samnewman
$ deploy Returns v456 Production
local
latest
Service Name
Version

28.
@samnewman
$ deploy Returns v456 Production
local
latest
Service Name
Version
Environment

29.
@samnewman
TestsBuild Large TestsUAT Prod
DB
Machine
UAT Environment
Machine
Perf

30.
@samnewman
TestsBuild UAT Perf
Master
DB
Machine
Perf Environment
Machine Machine Machine
Slave
DB
Perf Prod

31.
@samnewman
TestsBuild UAT Perf Prod
Master
DB
Machine
Production Environment
Machine Machine Machine
Slave
DB

32.
@samnewman
Same Artifact

33.
@samnewman
Same Artifact
Different Topology

34.
@samnewman
Core Principles?

35.
@samnewman
Independent Deployability
Core Principles?

36.
@samnewman
Independent Deployability
One Artifact For All Environments
Core Principles?

37.
@samnewman
Independent Deployability
One Artifact For All Environments
Same Deployment Process Everywhere
Core Principles?

38.
@samnewman
Artifacts
Core Principles
Platforms

39.
@samnewman
What do we want from an artifact?

40.
@samnewman
What do we want from an artifact?

41.
@samnewman
What do we want from an artifact?
Easy to create

42.
@samnewman
What do we want from an artifact?
Easy to deploy
Easy to create

43.
@samnewman
What do we want from an artifact?
Easy to deploy
Abstract out the tech stack
Easy to create

44.
@samnewman
What do we want from an artifact?
Easy to deploy
Abstract out the tech stack
Good for dev, good for ops
Easy to create

45.
@samnewman
Tarballs

46.
@samnewman
Tarballs
Giant bundles of stuff

47.
@samnewman
Tarballs
Easy to create
Easy to deploy
Abstract out the tech stack
Good for dev, good for ops

48.
@samnewman
Tarballs
Easy to create!
Easy to deploy
Abstract out the tech stack
Good for dev, good for ops

49.
@samnewman
Tarballs
Easy to create!
Easy to deploy"
Abstract out the tech stack
Good for dev, good for ops

50.
@samnewman
Tarballs
Easy to create!
Easy to deploy"
Abstract out the tech stack#
Good for dev, good for ops

51.
@samnewman
Tarballs
Easy to create!
Easy to deploy"
Abstract out the tech stack#
Good for dev, good for ops"

52.
@samnewman
Stack-specific

53.
@samnewman
Stack-specific
nuget
jar
pip
gems

54.
@samnewman
Stack-specific
Easy to create
Abstract out the tech stack
Easy to deploy
Good for dev, good for ops

55.
@samnewman
Stack-specific
Easy to create!
Abstract out the tech stack
Easy to deploy
Good for dev, good for ops

56.
@samnewman
Stack-specific
Easy to create!
Abstract out the tech stack
Easy to deploy#
Good for dev, good for ops

57.
@samnewman
Stack-specific
Easy to create!
Abstract out the tech stack"
Easy to deploy#
Good for dev, good for ops

58.
@samnewman
Stack-specific
Easy to create!
Abstract out the tech stack"
Easy to deploy#
Good for dev, good for ops"

59.
@samnewman
Everything in Go is Awesome

60.
@samnewman
Everything in Go is Awesome
FACT

61.
@samnewman
OS-Specific

62.
@samnewman
OS-Specific
$ sudo apt-get install myservice

63.
@samnewman
OS-Specific
$ sudo apt-get install myservice
$ deploy Returns v456 Production

64.
@samnewman
OS-Specific
Easy to deploy
Easy to create
Abstract out the tech stack
Good for dev, good for ops

65.
@samnewman
OS-Specific
Easy to deploy
Easy to create"
Abstract out the tech stack
Good for dev, good for ops

66.
@samnewman
OS-Specific
Easy to deploy!
Easy to create"
Abstract out the tech stack
Good for dev, good for ops

67.
@samnewman
OS-Specific
Easy to deploy!
Easy to create"
Abstract out the tech stack!
Good for dev, good for ops

68.
@samnewman
OS-Specific
Easy to deploy!
Easy to create"
Abstract out the tech stack!
Good for dev, good for ops#

69.
@samnewman
Build

70.
@samnewman
Build
Deb Repo

71.
@samnewman
Host
Build
Deb Repo

72.
@samnewman
Host
Build
Deb Repo

73.
@samnewman
Host
Build
Deb Repo

74.
@samnewman

75.
@samnewman
A
v1.5

76.
@samnewman
A
v2.1
A
v1.5

77.
@samnewman
A
v2.1
A
v1.5

78.
@samnewman
Host

79.
@samnewman
Host

80.
@samnewman
Host Host
Host Host

81.
@samnewman
Host Host
Host Host
Independent Execution
Environments FTW!

82.
@samnewman
Custom Images
Easy to deploy
Easy to create
Abstract out the tech stack
Good for dev, good for ops

83.
@samnewman
Custom Images
Easy to deploy
Easy to create#
Abstract out the tech stack
Good for dev, good for ops

84.
@samnewman
Custom Images
Easy to deploy#
Easy to create#
Abstract out the tech stack
Good for dev, good for ops

85.
@samnewman
Custom Images
Easy to deploy#
Easy to create#
Abstract out the tech stack
Good for dev, good for ops
!

86.
@samnewman
Custom Images
Easy to deploy#
Easy to create#
Abstract out the tech stack
Good for dev, good for ops
!
#

87.
@samnewman

88.
@samnewman

89.
@samnewman
Cost of isolated hosts is reduced…

90.
@samnewman
Cost of isolated hosts is reduced…
…in terms of effort…

91.
@samnewman
Cost of isolated hosts is reduced…
…in terms of effort…
…and computing resources

92.
@samnewman
Docker!
Easy to deploy
Abstract out the tech stack
Good for dev, good for ops
Easy to create

93.
@samnewman
Docker!
Easy to deploy
Abstract out the tech stack
Good for dev, good for ops
Easy to create#

94.
@samnewman
Docker!
Easy to deploy
Abstract out the tech stack
Good for dev, good for ops
Easy to create
!
#

95.
@samnewman
Docker!
Easy to deploy
Abstract out the tech stack
Good for dev, good for ops
Easy to create
!
!
#

96.
@samnewman
Docker!
Easy to deploy
Abstract out the tech stack
Good for dev, good for ops
Easy to create
!
!
#
#

97.
@samnewman
Artifacts
Core Principles
Platforms

98.
@samnewman
What do we want from our deployment
platform?

99.
@samnewman
What do we want from our deployment
platform?
Separate artifact from topology

100.
@samnewman
What do we want from our deployment
platform?
Separate artifact from topology
Makes handling lots of services easy!

101.
@samnewman
What do we want from our deployment
platform?
Separate artifact from topology
Makes handling lots of services easy!
Supports docker images

102.
@samnewman
Deployment Platforms?

103.
@samnewman
Docker Swarm

104.
@samnewman
Docker Swarm

105.
@samnewman
Docker Swarm

106.
@samnewman
Docker Swarm

107.
@samnewman
DOCKER SWARM

108.
@samnewman
DOCKER SWARM
Swarm Manager

109.
@samnewman
DOCKER SWARM
Swarm Node Swarm Node Swarm Node
Swarm Manager

110.
@samnewman
DOCKER SWARM
Swarm Node Swarm Node Swarm Node
Swarm Manager

111.
@samnewman
DOCKER SWARM
Swarm Node Swarm Node Swarm Node
Swarm Manager
$ docker …

112.
@samnewman
DOCKER SWARM
Swarm Node Swarm Node Swarm Node
Swarm Manager
$ docker …

113.
@samnewman
DOCKER SWARM
Swarm Node Swarm Node Swarm Node
Swarm Manager
$ docker …

114.
@samnewman
DOCKER SWARM
Swarm Node Swarm Node Swarm Node
Swarm Manager
$ docker …

115.
@samnewman
SCHEDULING STRATEGIES - BINPACK
Swarm Node Swarm Node Swarm Node
Swarm Manager

116.
@samnewman
SCHEDULING STRATEGIES - BINPACK
Swarm Node Swarm Node Swarm Node
Swarm Manager

117.
@samnewman
SCHEDULING STRATEGIES - SPREAD
Swarm Node Swarm Node Swarm Node
Swarm Manager

118.
@samnewman
SCHEDULING STRATEGIES - SPREAD
Swarm Node Swarm Node Swarm Node
Swarm Manager

119.
@samnewman
DOCKER COMPOSE
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/code
links:
- redis
redis:
image: redis

120.
@samnewman

121.
@samnewman
Doesn’t rebalance

122.
@samnewman
Doesn’t rebalance
Doesn’t restart failed containers

123.
@samnewman
Doesn’t rebalance
Doesn’t restart failed containers
Plays nice with the rest of docker

124.
@samnewman
Doesn’t rebalance
Doesn’t restart failed containers
Plays nice with the rest of docker
Case studies thin on the ground

125.
@samnewman

126.
@samnewman
Mesos Master
MESOS

127.
@samnewman
Mesos Master
MESOS

128.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS

129.
@samnewman
Frameworks!

130.
@samnewman

131.
@samnewman
Frameworks

132.
@samnewman
Frameworks
Scheduler

133.
@samnewman
Frameworks
Scheduler Executor

134.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS

135.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS
Hadoop Scheduler

136.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS
Hadoop
Executor
Hadoop Scheduler

137.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS
Hadoop
Executor
Hadoop Scheduler

138.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS
Hadoop
Executor
Hadoop Scheduler
Hadoop

139.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS
Hadoop
Executor
Hadoop Scheduler
Hadoop
Marathon
Marathon

140.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS
Hadoop
Executor
Hadoop Scheduler
Hadoop
Marathon
Marathon
Marathon Scheduler

141.
@samnewman
Mesos Master
Mesos Agent Mesos Agent Mesos Agent
MESOS
Hadoop
Executor
Hadoop Scheduler
Hadoop
Marathon
Marathon
Marathon Scheduler

142.
@samnewman
OTHER FRAMEWORKS…
https://github.com/alde/eremetic

143.
@samnewman
AWS Lambda

144.
@samnewman

145.
@samnewman
Great if you need to run other workloads

146.
@samnewman
Great if you need to run other workloads
Really powerful - fully featured, widely used

147.
@samnewman
Great if you need to run other workloads
Really powerful - fully featured, widely used
Fairly complex - lots of moving parts

148.
@samnewman

149.
@samnewman

150.
@samnewman
KUBERNETES ARCHITECTURE…SORT OF

151.
@samnewman
API Server
KUBERNETES ARCHITECTURE…SORT OF

152.
@samnewman
API Server
KUBERNETES ARCHITECTURE…SORT OF

153.
@samnewman
API Server
Kubelet Kubelet Kubelet
KUBERNETES ARCHITECTURE…SORT OF

154.
@samnewman
API Server
Kubelet Kubelet Kubelet
$ kubectl …
KUBERNETES ARCHITECTURE…SORT OF

155.
@samnewman
API Server
Kubelet Kubelet Kubelet
$ kubectl …
KUBERNETES ARCHITECTURE…SORT OF

156.
@samnewman
PODS?

157.
@samnewman
A collection of tightly coupled
containers, running on one node
PODS?

158.
@samnewman
A collection of tightly coupled
containers, running on one node
PODS?
Can have metadata, volumes too

159.
@samnewman
A collection of tightly coupled
containers, running on one node
PODS?
Can have metadata, volumes too
Pods are mortal - not long running!

160.
@samnewman
A collection of tightly coupled
containers, running on one node
PODS?
Can have metadata, volumes too
Pods are mortal - not long running!
A pod = a unit of scheduling

161.
@samnewman
SERVICES!

162.
@samnewman
SERVICES!
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "my-service"
},
"spec": {
"selector": {
"app": "MyApp"
},
"ports": [
{
"protocol": "TCP",
"port": 80,
"targetPort": 9376
}
]
}
}

163.
@samnewman
SERVICES!
A mapping of
metadata and ports
to a set of pods
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "my-service"
},
"spec": {
"selector": {
"app": "MyApp"
},
"ports": [
{
"protocol": "TCP",
"port": 80,
"targetPort": 9376
}
]
}
}

164.
@samnewman
JSON is neither human readable nor human writeable

165.
@samnewman
JSON is neither human readable nor human writeable
Please stop using it for anything other than machines

166.
@samnewman
Kubelet

167.
@samnewman
Kubelet

168.
@samnewman
Kubelet

169.
@samnewman
Kubelet
Service Proxy

170.
@samnewman
Kubelet
Service Proxy
:80

171.
@samnewman
You don’t scale a service…
Kubelet
Service Proxy
:80

172.
@samnewman
You don’t scale a service…
…you scale the pods!
Kubelet
Service Proxy
:80

173.
@samnewman

174.
@samnewman
Simpler to setup than Mesos - but more single purpose too

175.
@samnewman
Simpler to setup than Mesos - but more single purpose too
Closer to a PAAS

176.
@samnewman
Simpler to setup than Mesos - but more single purpose too
Closer to a PAAS
Pods can be confusing!

177.
@samnewman
Simpler to setup than Mesos - but more single purpose too
Closer to a PAAS
Fairly new, but a growing amount of impressive support
Pods can be confusing!

178.
@samnewman
Docker Swarm

179.
@samnewman
Docker Swarm

180.
@samnewman
Docker Swarm

181.
@samnewman
Docker Swarm

182.
@samnewman

183.
@samnewman
Core Principles

184.
@samnewman
Core Principles
Independent
Deployability

185.
@samnewman
Core Principles
One Artifact For All
Environments
Independent
Deployability

186.
@samnewman
Core Principles
One Artifact For All
Environments
Independent
Deployability
Same
Deployment
Process

187.
@samnewman
Core Principles
One Artifact For All
Environments
Independent
Deployability
Same
Deployment
Process
Docker Images As Artifacts

188.
@samnewman
Core Principles
One Artifact For All
Environments
Independent
Deployability
Same
Deployment
Process
Docker Images As Artifacts
Criteria For Selecting A Platform

189.
@samnewman

190.
@samnewman
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
AUTHD

191.
@samnewman
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
AUTHD
Send me your questions on twitter!

192.
@samnewman
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
AUTHD
Send me your questions on twitter!
@samnewman

193.
@samnewman
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
AUTHD
Send me your questions on twitter!
@samnewman
And I’ll write up an answer for you!

194.
@samnewman
snewman@thoughtworks.com
THANKS!