This talk was presented at the Montreal Kubernetes Meetup on 11/8/17 by Richard Li from Datawire.

1. Developer-first workflows (for
microservices) on Kubernetes
Richard Li

2. datawire.io
● Building a cloud application using
microservices in 2013
● Distributed systems engineers
● Multiple services
● Prototyping was really fast
● … then velocity slowed down by a lot
when we started getting users and
needing to add more features
2
Microservices at Datawire ...

3. datawire.io
Part 1: How do I gain the
benefits of microservices?
(How do I break up my monolith?)
(How do I architect my app with microservices?)
(What infrastructure do I need in place before I
can benefit from microservices?)
3

4. datawire.io 4
Stability/Maturity
Velocity
Prototype Production Mission critical

5. datawire.io 5
Stability/Maturity
Velocity
Prototype Production Mission critical

6. datawire.io
How do I break up my monolithic architecture?
How do I break up my monolithic process into
multiple independent processes?
6

7. datawire.io 7
Define
Code
Test
Release
Prod

8. datawire.io 8
Define
Code
Test
Release
Prod
Centralized process
● Specialized teams
● Fixed policies (e.g.,
release criteria)

9. datawire.io 9
Microservices lets you run multiple processes!

10. datawire.io 10
Industrial Revolution.

11. datawire.io
Microservices is a distributed
development architecture
workflow.
11

12. datawire.io 12
Stability/Maturity
Velocity
Prototype Production Mission critical
● How do I get to Continuous Deployment incrementally?
● How do I limit the scope of PCI (audit process)?
● How do I ship feature X as fast as possible?

13. datawire.io
Microservices is ...
● Multiple workflows
○ Including your existing workflow!
○ Workflows designed for different
stability/velocity tradeoffs
● Simultaneous workflows
13

14. datawire.io
Doing things this way shifts how people operate!
● Specialists become generalists → design better holistic systems
● Dev vs Ops vs QA → roadmap / API conversations
● Challenging for engineers → More fun!
14

15. datawire.io
Part 2: How do I create multiple,
simultaneous workflows?
15

16. datawire.io
Create self-sufficient, autonomous
software teams.
16

17. datawire.io
Why self-sufficiency and autonomy?
● Self-sufficient
○ Team does not need to rely on other teams to achieve its goals
● Autonomy
○ Team is able to independently make (process) decisions on how to achieve its goals
17

18. datawire.io
Centralized specialist functions are a common barrier
to self-sufficiency and autonomy
18
Centralized architecture
Centralized infrastructure / ops*
(You might need a platform team,
eventually)

19. datawire.io 19

20. datawire.io 20
Monolith

21. datawire.io 21
Monolith
(soon-to-be) self-sufficient,
autonomous team

22. datawire.io
I’m stranded on API island. Do I have to
build my workflow from scratch?
22

23. datawire.io
No! Kubernetes / Docker / Envoy give you the infra you
need
23

24. datawire.io
Docker
● A consistent build/runtime for your
service
● Run locally or in Kubernetes cluster
● Massive community & ecosystem
○ 14M Docker hosts
○ 3300+ contributors
○ Lots of third party tools
24

25. datawire.io
Kubernetes
● Cloud POSIX
● Declarative definition of your cloud
infrastructure
● Supported by all major vendors:
Google, Microsoft, Mesos, Docker,
Oracle, Red Hat, IBM all support it
● … except for Amazon.
○ Yet 62% of K8S workloads on AWS
25

26. datawire.io
Envoy
● Modern L7 proxy, designed for cloud
applications
○ Resilience: global rate-limiting, circuit
breakers
○ APIs for managing fleets of Envoys
○ L7 observability (incl distributed tracing)
● Fast growing ecosystem &
community
○ Maintained by Lyft
○ Google, IBM, Apple, Datawire, Verizon
○ CNCF project
26

27. datawire.io
Cloud Native Compute Foundation
27
● Istio
● SPIFFE
● App dev

28. datawire.io
Summary
1. Microservices is a distributed workflow: multiple & simultaneous processes.
2. Start building your distributed workflow by creating a self-sufficient, autonomous
team.
3. Kubernetes / Docker / Envoy give you the basic operational primitives you need.
28

29. datawire.io
Part 3: How do I actually use these
technologies to build my workflow?
29

30. datawire.io 30
Monolith
Prototype new feature, FAST

31. datawire.io 31
Stage 1: Prototyping
Existing workflow: Designed for
mature app, not prototyping
Challenge: Prototyping workflow that
is fast but not one-off
Strategies:
● Dev infra ~= Prod infra
● Self-service operational
infrastructure

32. datawire.io
Strategy: Dev infrastructure ~= Prod infrastructure
32
● End-to-end
○ Have developers access the service the same way as end users (e.g., through an
API Gateway, ingress, etc.)
● Environmental consistency
○ Insure that your dev runtime is the same as your initial production runtime
○ Environmental differences between dev & prod will cause bugs. So eliminate
them upfront.
Your dev infrastructure should resemble your production
infrastructure as much as possible.

33. datawire.io
Strategy: Self-service operational infrastructure
33
Infrastructure needs to be configured by the service team, not by
operations (otherwise, there’s a bottleneck.)
Service team needs to be able to
publish new services, without
operations.
Service team has access to the
Kubernetes manifest (maybe they
use a standard template).

34. datawire.io
Implementing Stage 1 strategies
● Eliminate environmental differences
○ Containerize your service and use the same container for dev & prod
○ Template your Kubernetes manifests so you can use the same manifest for dev & prod
● End-to-end
○ Use an API Gateway for dev and prod (you can even use the same one)
● Self-service operational infrastructure
○ Expose the right configuration options, in a rational way, for your infrastructure to the service
team
■ Kubernetes manifest
■ Configuring your API Gateway
34

35. datawire.io 35
Monolith
Now in production, and need to
ITERATE

36. datawire.io 36
Stage 2: Production users
Existing workflow: App-level
integration testing & roll out;
dedicated QA team; unit tests
Challenge: Update service quickly,
while not letting bugs impact the end
user
Strategies:
● Software redundancy

37. datawire.io
Strategy: Multiple versions for software redundancy
37
End user
Primary version
Canary version
Dev version

38. datawire.io 38
Stage 3 Internal service consumption
Existing workflow: None
Challenge: Communicate to
consumers the service contract, and
support that contract in the service
Strategy:
● Service level objectives
● Network-level observability &
resilience

39. datawire.io
● Define the target level of your service as part of your API
● Common SLOs relate to throughput, latency, and availability
○ This API will support up to 5,000 requests per second per consumer
○ The expected latency at p50, p90, p99
● Simple implementation of SLOs can be via monitoring
Strategy: Service level objectives
39

40. datawire.io
Strategy: Network level observability & resilience
● Observability
○ Figure out which service is causing the problem
○ Need visibility at a network level
○ Distributed tracing (e.g., Jaeger, Zipkin)
● Resilience
○ Avoid cascade failure
○ Network-level rate limiting, circuit breaking, etc.
● Service mesh
○ Strategy for implementing network-level observability & resilience that’s transparent to end users
○ Istio just proposed to the CNCF
40

41. datawire.io
Summary
1. Microservices is a distributed workflow.
2. Start building your distributed workflow by creating a self-sufficient, autonomous
team.
3. Kubernetes / Docker / Envoy give you the basic operational primitives you need.
4. Your workflow requirements depend on your service maturity (prototype,
production, internal dependencies).
5. Strategies for building your workflow
a. Prototype: Dev ~= Prod, self-service operational infrastructure
b. Production: Software redundancy
c. Internal dependencies: Service level objectives, network-level observability/resilience
41

42. datawire.io
So how does this work in practice?
42

43. datawire.io
Your traditional monolith workflow ...
● Intended for iterative improvement & prevent downtime
○ Code locally in dev environment
○ Pull request
○ Branch for release
○ QA branch & integration test on staging environment
○ Release on production environment
● Different environments, specialized teams
43

44. datawire.io
Versus microservice infrastructure & workflow ...
● Eliminate environmental differences
○ Container -- same run time environment
○ Kubernetes -- same deployment environment
● Self-service operational infrastructure
○ Templated Kubernetes manifest
○ Self-service API Gateway
● Multiple versions for software redundancy
○ Facilitated by API Gateway / router
44
Will let you prototype / launch / update a microservice with a
workflow that’s tailored to feature development.

45. datawire.io
Thank you!
● richard@datawire.io
● Getting ready to launch hands-on tutorials on building your own distributed
workflow. If you’re interested in being notified, sign up here:
○ https://d6e.co/montrealk8s
● If you’re interested in any of our open source tools, check them out:
○ https://forge.sh for deployment
○ https://www.telepresence.io for real-time live coding
○ https://www.getambassador.io self-service API Gateway built on Envoy
45

46. END