As your service footprint grows, adding traffic control capabilities beyond stock solutions like kube-proxy becomes critical. Envoy provides fine grained routing control, load shedding, and metrics that help you scale your environment smoothly. We'll walk through several traffic control strategies using Envoy.
2. Why Care about Traffic Control
Generaliza5ons: Kubernetes leads to a bunch of good things.
• Crea5ng new services is easier.
• Deploying new service versions is easier.
• Deploying smaller services is easier.
3. Why Care about Traffic Control
But the good things aren’t free.
• New code needs to be (safely!) integrated with your request flow.
• Addi5onal abstrac5ons have < 100% reliability.
• Longer call chains introduce more chances for failure.
4. Goals of Traffic Control
• Resilience
• Distributed systems are never “up”1. Dealing with failures should be
straighTorward.
• Rou5ng
• Introducing a new code to the call chain is a common opera5on. It
should be straighTorward.
1. Charity Majors, hYps://opensource.com/ar5cle/17/7/state-systems-administra5on
5. The Setup
• Create scenarios using augmented Envoy examples
• Use wrk to drive load against the system and measure results
• Curl, because no demo is complete without some curl
• A preview of envoy-tools to observe Envoy stats directly
6. Control Requires Visibility
• Making unobservable changes is not advised.
• Envoy comes with great tools out of the box.
• Stats on listeners, clusters, protocols, and more.
• An admin server for direct observa5on and control.
• envoy-tools (coming soon!) – a repository of tools that provide a more
approachable interface.
10. Retries
Envoy supports retry policies aYached to
routes
• Select error codes to retry on.
• Configure 5meouts for each retry.
• Configure number of retries.
13. Safe Retries
• Usually you don’t want to retry all
requests.
• Side effects are important to consider.
• Atomicity is important to consider.
• Computa5onal expense is important to
consider.
• Add more routes, and configure retries
accordingly.
14. Load Shedding
• Some5mes you get more traffic than you can handle.
• Envoy supports request limits on a per-cluster basis.
• Envoy also supports two priority groups, allowing you to save slots for
important traffic.
15. Without Circuit Breakers
Failures are fine, but 99% latency is slowwwwww
as requests just back up
Also, POST requests are totally offline because
we’re swamped with GETs
17. With Circuit Breakers
Gobs of failures, but p99 latency is s5ll good.
Also, POST requests are available.
Also, we told clients to back off with the
x-envoy-overloaded response header.
19. An Overview of Rou5ng
• Endpoint metadata for richer rou5ng primi5ves
• Probabilis5c distribu5on of traffic across mul5ple clusters
• 1% of traffic to my-great-rewrite, 99% to legacy
• 1% of traffic to v2 of my service, 99% to v1
• Header based rou5ng to cluster subsets
• If “x-canary” is set route to endpoints with a version label of v2
• Priority rou5ng, which we saw in the circuit breaking example
• Zone aware rou5ng
21. Header-based Canary
• When we specify the canary header, the route matches and we (and
only we) are routed to service1a
• When header is not present, the route doesn’t match and we go on to
the next route, sending traffic to service1
22. Probabilis5c Rollout
• With the run5me match, we choose this route 25% of the 5me,
sending 25% of our traffic to service1
24. The Easy Way
• Restar5ng servers on every config change is tedious in this demo.
• It’s even more tedious in produc5on.
• Envoy provides a beYer way—the xDS APIs.
25. xDS APIs
• CDS - discover clusters, which are logical groupings of endpoints.
• A cluster defini5on can have a reference to an EDS endpoint
• EDS - discover endpoints for a cluster.
• LDS - discover listeners for an Envoy
• A listener’s filter chain can have a reference to an RDS endpoint
• RDS - discover routes for a filter chain
26. Dynamic Config
• The xDS APIs give you a central point-of-control to manage a fleet of
Envoys
• Bridge service discovery (e.g. from Kubernetes) to Envoy
• Bridge rou5ng config (e.g. from Houston) to Envoy
27. Advanced Rou5ng with EDS
• CDS (cluster discovery service) defines groups of endpoints.
• EDS (endpoint discovery service) discovers the actual endpoints for
clusters.
• EDS allows you to aYach metadata to an endpoint.
• Our mul5-cluster example can be collapsed to a metadata based
approach on a single cluster.
28. Even Easier with Houston
• An CDS/EDS server with integra5ons to EC2, ECS, Kubernetes,
Consul, DC/OS, or JSON files
• An LDS/RDS server with an intui5ve route configura5on UI
• Stats parsing, forwarding, and change tracking
