Keynote delivered by Onsi Fakhouri, Engineering Manager at Pivotal.
Diego is a ground-up rewrite of the DEA - a major component of the Cloud Foundry Elastic Runtime. This talk will motivate the need for Diego, the philosophy behind Diego, and present a few choice technical details to illustrate some of the more interesting ideas we've been playing with.
4. Cloud
Controller
What is being rewritten?
Stage App
Run n App Instances
(and keep them running)
http://âŠ
Push App
> cf
Route to App
5. DEA Pool (Droplet Execution Agent)
What is being rewritten?
http://âŠ
Push App
> cf Cloud
Controller
Router
(API)
6. What is being rewritten?
http://âŠ
Push App
> cf Cloud
Controller
Router
DEA Pool (Droplet Execution Agent)
(API)
7. What is being rewritten?
http://âŠ
Push App
> cf Cloud
Controller
Router
DEA Pool (Droplet Execution Agent)
DEA
Staging Apps
Running Apps(API)
8. What is being rewritten?
http://âŠ
Push App
> cf Cloud
Controller
Router
DEA Pool (Droplet Execution Agent)
DEA
Staging Apps
Running Apps
Warden
Containerization
(API)
9. What is being rewritten?
http://âŠ
Push App
> cf Cloud
Controller
Router
DEA Pool (Droplet Execution Agent)
DEA
Staging Apps
Running Apps
Warden
Containerization
Health
Manager
(API)
10. What is being rewritten?
Push App
http://âŠ
> cf Cloud
Controller
Router
Health
Manager
DEA Pool (Droplet Execution Agent)
DEA
Staging Apps
Running Apps
Warden
Containerization
NATS
(message bus)
(API)
11. What is being rewritten?
Push App
http://âŠ
> cf Cloud
Controller
Router
Health
Manager
DEA Pool (Droplet Execution Agent)
DEA
Staging Apps
Running Apps
Warden
Containerization
NATS
(message bus)
(API)
28. Why rewrite?
Cloud
Controller
Tight Coupling
Poor separation
of concerns
Or
ch
es
tr
at
ion
> cf scale
DEA
Warden
DEA
Warden
DEA
Warden
DEA
Warden
DEA
Warden
DEA
Warden
start
fails
Too much responsiblity
29. Why rewrite?
Tight Coupling
Poor separation
of concerns
Cloud
Controller
DEA
Warden
DEA
Warden
DEA
Warden
DEA
Warden
DEA
Warden
DEA
Warden
Triangular
Dependencies
30. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
DEA
Warden
Cloud
Controller
DEA
Warden
31. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
DEA
Warden
Cloud
Controller
DEA
Warden
When itâs time to
upgrade the DEAs
When itâs time to
upgrade the DEAs
32. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
DEA
Warden
Cloud
Controller
DEA
Warden
When itâs time to
upgrade the DEAs
we perform a rolling deploy
33. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
DEA
Warden
DEA
Warden
Cloud
Controller
34. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
DEA
Warden
DEA
Warden
Cloud
Controller
âbye!â
35. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
âbye!â
DEA
Warden
DEA
Warden
36. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
37. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
38. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
39. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
all clear!
40. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
all clear!
41. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
all clear!
Problematic
42. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
all clear!
Problematic
43. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
all clear!
Problematic
??
??
44. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
all clear!
Problematic
45. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
all clear!
Problematic
46. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
start!
all clear!
Problematic
47. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
all clear!
Problematic
start!
48. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
all clear!
Problematic
start!
49. Why rewrite?
Tight Coupling
Poor separation
of concerns
Triangular
Dependencies
Health
Manager
DEA
Warden
DEA
Warden
Cloud
Controller
start!
âbye!â
DEA
Warden
DEA
Warden
all clear!
Problematic
start!
51. Why rewrite?
Tight Coupling Poor separation
of concerns
Triangular
Dependencies
Or
ch
es
tr
at
ion
complex interactions
52. Why rewrite?
Tight Coupling Poor separation
of concerns
Triangular
Dependencies
Or
ch
es
tr
at
ion
hard to test
complex interactions
53. Why rewrite?
Tight Coupling Poor separation
of concerns
hard to test
hard to reason through
complex interactions
Triangular
Dependencies
Or
ch
es
tr
at
ion
65. Why rewrite?
Platform SpeciïŹc
Domain SpeciïŹc
(app, app, app, app)
Tight Coupling Poor separation
of concerns
Or
ch
es
tr
at
ion
Triangular
Dependencies
Hard
to add new features
to maintain existing features
67. Show me Diego
Strong concurrency support
Written in Golang
Strongly typed
Explicit error handling
Promotes developer discipline
Strong low-level OS support
68. Show me Diego
Domain SpeciïŹc
(app, app, app, app)
One-oïŹ Tasks
(guaranteed to only run once)
Long Running Processes
(n monitored instances)
The Right(?)
Abstraction
72. Cloud
Controller
Executor Pool
Show me Diego
The Right(?) Abstraction
Run
Tasks
Launch
Long Running
Processes
App-Manager
Run App Launch LRP
Stager
Stage App Run Task
73. Cloud
Controller
Executor Pool
Show me Diego
The Right(?) Abstraction
App-Manager
Run App Launch LRP
Run
Tasks
Launch
Long Running
Processes
Stager
Stage App Run Task
Express speciïŹc domain
74. Cloud
Controller
Executor Pool
Show me Diego
The Right(?) Abstraction
App-Manager
Launch LRP
Run
Tasks
Launch
Long Running
Processes
Stager
Run Task
Express speciïŹc domain
In terms of generic recipes
Run App
Stage App
75. Cloud
Controller
Executor Pool
Show me Diego
The Right(?) Abstraction
App-Manager
Stager
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Launch LRP
Run Task
Run App
Stage App
76. Cloud
Controller
Executor Pool
Show me Diego
The Right(?) Abstraction
App-Manager
Stager
Express speciïŹc domain
In terms of generic recipes
Exec
Recipes
Exec
Run
Tasks
Launch
LRPs
Rep
Launch LRP
Run Task
Run App
Stage App
77. Cloud
Controller
Executor Pool
Show me Diego
The Right(?) Abstraction
App-Manager
Stager
Express speciïŹc domain
In terms of generic recipes
Exec
Recipes
Exec Garden
Manage
Containers
Run
Tasks
Launch
LRPs
Rep
Launch LRP
Run Task
Run App
Stage App
78. Cloud
Controller
Executor Pool
Show me Diego
The Right(?) Abstraction
App-Manager
Stager
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Exec
Recipes
Exec Garden
Manage
Containers
Linux
Backend
Run
Containers
Launch LRP
Run Task
Run App
Stage App
79. Cloud
Controller
Executor Pool
Show me Diego
App-Manager
Stager
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Exec
Recipes
Exec Garden
Manage
Containers
Linux
Backend
Run
Containers
GenericSpeciïŹc
Launch LRP
Run Task
Run App
Stage App
80. Cloud
Controller
Executor Pool
Show me Diego
App-Manager
Stager
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Exec
Recipes
Exec Garden
Manage
Containers
Linux
Backend
Run
Containers
GenericSpeciïŹc
Launch LRP
Run Task
Run App
Stage App
New features go here!
(e.g. cron-like tasks)
81. Cloud
Controller
Executor Pool
Show me Diego
App-Manager
Stager
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Exec
Recipes
Exec Garden
Manage
Containers
Linux
Backend
Run
Containers
GenericSpeciïŹc
Flexibility
Launch LRP
Run Task
Run App
Stage App
New features go here!
(e.g. cron-like tasks)
83. Show me Diego
Platform Independent â
Cloud
Controller
Executor Pool
App-Manager
Run App Launch LRP
Stager
Stage App Run Task
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Exec
Recipes
Exec Garden
Manage
Containers
Linux
Backend
Run
Containers
84. Cloud
Controller
Executor Pool
App-Manager
Run App Launch LRP
Stager
Stage App Run Task
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Exec
Recipes
Exec Garden
Manage
Containers
Linux
Backend
Run
Containers
Show me Diego
Platform Independent â
â â
â
â â â
85. Cloud
Controller
Executor Pool
App-Manager
Run App Launch LRP
Stager
Stage App Run Task
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Exec
Recipes
Exec Garden
Manage
Containers
Linux
Backend
Run
Containers
Show me Diego
â â
â
â â â
Platform Independent â
106. complex interactions
hard to test
hard to reason through
Show me Diego
simulation driven
Cloud
Controller
Executor Pool
App-Manager
Run App Launch LRP
Stager
Stage App Run Task
Express speciïŹc domain
In terms of generic recipes
Run
Tasks
Launch
LRPs
Rep
Exec
Recipes
Exec Garden
Manage
Containers
Linux
Backend
Run
Containers
107. Show me Diego
executor
rep
stager
14 small single-responsibility
components!
app-manager
auctioneer
converger
etcd-metrics-server
etcd
ïŹle-server
garden
linux-circus
metricz
route-emitter
tps
simulation driven
complex interactions
hard to test
hard to reason through
108. Show me Diego
executor
rep
stager app-manager
auctioneer
converger
etcd-metrics-server
etcd
ïŹle-server
garden
linux-circus
metricz
route-emitter
tps
â
â
â
â â
â
â â
â
ââ
â
â
â
unit-testedâ
simulation driven
complex interactions
hard to test
hard to reason through
109. Show me Diego
executor
rep
stager app-manager
auctioneer
converger
etcd-metrics-server
etcd
ïŹle-server
garden
linux-circus
metricz
route-emitter
tps
â
â
â
â â
â
â â
â
ââ
â
â
â
?unit-testedâ
simulation driven
complex interactions
hard to test
hard to reason through
110. Show me Diego
repâ
garden â
linux-circusâ
auctioneerâ metriczâ
route-emitterâ
stagerâ app-managerâ
executorâ
ïŹle-server â
tpsâ
etcdâ
converger â
etcd-metrics-serverâ
unit-testedâ
simulation driven
Actors
complex interactions
hard to test
hard to reason through
111. Show me Diego
unit-testedâ
simulation driven
Diego is a play
Actors
repâ
garden â
linux-circusâ
auctioneerâ metriczâ
route-emitterâ
stagerâ app-managerâ
executorâ
ïŹle-server â
tpsâ
etcdâ
converger â
etcd-metrics-serverâ
complex interactions
hard to test
hard to reason through
112. Show me Diego
repâ
garden â
linux-circusâ
auctioneerâ
metriczâ
route-emitterâ
stagerâ
app-managerâ
executorâ
ïŹle-server â
tpsâ
etcdâ
converger â
etcd-metrics-serverâ
communication and role
encoded via
shared library
script
shared narrative
unit-testedâ
simulation driven
Diego is a play
Actors
complex interactions
hard to test
hard to reason through
114. Show me Diego
complexity in a distributed system
of this scope is
real and necessary
Diego embraces this and tries
to make its complexity:
explicit
transparent
⎠easier to reason about
integration tests â
shared narrative
unit-testedâ
simulation driven
complex interactions
hard to test
hard to reason through
115. Show me Diego
ïŹexible abstraction
extensible
robust
agile
Tasks/LRPs
Platform-Independent
SELFManaging
Handle on Complexity