An API gateway is at the core of how APIs are managed, secured, and presented within any web-based system. Although the technology has been in use for many years, it has not always kept pace with recent developments within the cloud native space, and many engineers are confused about how a cloud native API gateway relates to Kubernetes Ingress or a Service load balancer. Join this session to learn about: The evolution of API gateways over the past ten years, and how the original problems they were solving have shifted in relation to cloud native technologies and workflow Current challenges of using an API gateway within Kubernetes: scaling the developer workflow; and supporting multiple architecture styles and protocols Strategies for exposing Kubernetes services and APIs at the edge of your system A brief guide to the (potential) future of cloud native API gateways

1. @danielbryantuk | @ambassadorlabs1
TRACK: MODERN INFRASTRUCTURE
NOVEMBER 12, 2020
Daniel Bryant
The Past, Present,
and Future of Cloud
Native API Gateways

2. @danielbryantuk | @ambassadorlabs
tl;dr
• API gateways have undergone a series of evolutions,
driven by architecture and technology
• Adopting microservices, Kubernetes, and cloud changes the workflow
• Choose your Kubernetes API gateway (and platform) intentionally
2

3. @danielbryantuk | @ambassadorlabs
@danielbryantuk
3

4. @danielbryantuk | @ambassadorlabs4
Edge: The boundary between your
data center and your user(s)

5. @danielbryantuk | @ambassadorlabs5
Thesis:
The evolution of the edge has been driven by
application architecture and technology

6. @danielbryantuk | @ambassadorlabs6
~1995

7. @danielbryantuk | @ambassadorlabs
Application Architecture in the ‘90s
7

8. @danielbryantuk | @ambassadorlabs
Hardware Load Balancer
8
User Systems administrators
Purpose High availability / scalability
Key Features
Load balancing
(round robin, sticky sessions)
Health checks

9. @danielbryantuk | @ambassadorlabs9
~2000

10. @danielbryantuk | @ambassadorlabs
Similar application architecture
10

11. @danielbryantuk | @ambassadorlabs11
20022001

12. @danielbryantuk | @ambassadorlabs
Software Load Balancer
12
User
Systems administrators
(“pre DevOps”)
Purpose High availability / scalability
Key Features
Load balancing
Health checks
Observability

13. @danielbryantuk | @ambassadorlabs13
~2005

14. @danielbryantuk | @ambassadorlabs14

15. @danielbryantuk | @ambassadorlabs15
Ecommerce
0
75
150
225
300
2000 2005 2010 2015 2019
Amazon Revenue,
Billions

16. @danielbryantuk | @ambassadorlabs16

17. @danielbryantuk | @ambassadorlabs
The Application Delivery Controller (ADC)
17

18. @danielbryantuk | @ambassadorlabs
Application Delivery Controllers (ADCs)
18
User Systems administrators
Purpose
High availability and
application acceleration
Key Features
SSL offload, caching,
compression + load balancing

19. @danielbryantuk | @ambassadorlabs19
~2010

20. @danielbryantuk | @ambassadorlabs
The proliferation of APIs
20
2005: API launched 2008 2009

21. @danielbryantuk | @ambassadorlabs
API Gateway (1st Gen)
21
User
Systems administrators &
API developers
Purpose
Expose business APIs to broader
ecosystem (“API management”)
Key Features
L7 routing (e.g., throttling),
Publishing, Dev Portal, Analytics,
Monetization

22. @danielbryantuk | @ambassadorlabs22
~2015

23. @danielbryantuk | @ambassadorlabs23

24. @danielbryantuk | @ambassadorlabs
Mini-services
24

25. @danielbryantuk | @ambassadorlabs
API Gateway (2nd Generation)
25

26. @danielbryantuk | @ambassadorlabs
API Gateway (2nd Generation)
26
User
Systems administrators &
API developers
Purpose
Centralize cross-cutting app
concerns
Key Features
Authentication, rate limiting,
monitoring + routing

27. @danielbryantuk | @ambassadorlabs27
Building
Cloud-Native
Applications

28. @danielbryantuk | @ambassadorlabs
Cloud-Native Microservices
28
• Modularisation (“microservices”)
• Built, released, & operated by
independent application teams
• Scaled independently

29. @danielbryantuk | @ambassadorlabs
App Architecture: A Spectrum of Services
• Different locations (K8s, VMs, FaaS)
• Different protocols (gRPC, HTTP, WebSockets, TCP)
• Different load balancing requirements
(sticky sessions, round robin)
• Different authentication requirements
29

30. @danielbryantuk | @ambassadorlabs
Cloud Gateway
30
Real-time Service
Discovery
Need ADC-like traffic management
capabilities: timeouts, retries, rate
limiting, load balancing, caching, …
Need API Gateway-type
management capabilities:
authentication, developer
portal, metrics, …
1
2
3

31. @danielbryantuk | @ambassadorlabs
Microservices lead to an even bigger change.
31

32. @danielbryantuk | @ambassadorlabs32
“You build it, you run it”
- Werner Vogels, CTO Amazon
i.e. you own what you code,
from idea to production

33. @danielbryantuk | @ambassadorlabs
Workflow: Full Cycle Development
33
• App teams have full responsibility (and authority) for
delivering a service, and ultimately, value to users
• Increase agility by accelerating the feedback loop
• https://netflixtechblog.com/full-cycle-developers-at-netflix-a08c31f83249

34. @danielbryantuk | @ambassadorlabs34
This is a change in workflow.

35. @danielbryantuk | @ambassadorlabs35
Thesis:
The future evolution of the edge will be driven by
application architecture, technology, and workflow

36. @danielbryantuk | @ambassadorlabs36
Two Biggest
Challenges with
k8s & the Edge

37. @danielbryantuk | @ambassadorlabs
Challenge #1: Scaling Edge Management
37
Development
Team
Operations / Platform Team

38. @danielbryantuk | @ambassadorlabs
Challenge #1: Scaling Edge Management

39. @danielbryantuk | @ambassadorlabs
Challenge #2: Supporting Diverse Edge Requirements
39

40. @danielbryantuk | @ambassadorlabs40
Three Strategies
for K8s
Gateways

41. @danielbryantuk | @ambassadorlabs
Three Strategies for the Edge with Kubernetes
#1: Deploy an Additional Kubernetes API Gateway
#2: Extend Existing API Gateway
#3: Deploy an in-Cluster Edge Stack
41
https://www.getambassador.io/resources/strategies-managing-apis-edge-kubernetes/

42. @danielbryantuk | @ambassadorlabs
#1 Deploy an Additional Kubernetes API Gateway
• Simply deploy an additional “in-cluster” gateway
• Below the existing gateway
• Below the load balancer
• Management
• Development teams responsible
• OR existing ops team manages this
42

43. @danielbryantuk | @ambassadorlabs
#1 Deploy an Additional Kubernetes API Gateway
• Pros
• There is minimal change to the core edge infrastructure.
• Incremental migration easily
• Cons
• Increased management overhead of working with
different components
• Challenging to expose the functionality to each
independent microservice teams
43

44. @danielbryantuk | @ambassadorlabs
#2 Extend Existing API Gateway
• Implemented by modifying or augmenting the existing API
gateway solution
• Enable synchronization between the API endpoints and
location of k8s services
• Custom ingress controller for the existing API Gateway or
load balancer
44

45. @danielbryantuk | @ambassadorlabs
#2 Extend Existing API Gateway
• Pros
• Reuse the existing tried and trusted API gateway
• Leverage existing integrations with on-premises
infrastructure and services
• Cons
• Workflows must change to preserve a single source of
truth for the API gateway configuration.
• Limited amount of configuration parameters via
Kubernetes annotations
45

46. @danielbryantuk | @ambassadorlabs
#3 Deploy an In-Cluster Edge Stack
• Deploy Kubernetes-native API gateway with
integrated supporting edge components
• Installed in each of the new Kubernetes clusters,
replacing existing edge
• Ops team own, and provide sane defaults
• Dev teams responsible for configuring the edge
stack as part of their normal workflow
46

47. @danielbryantuk | @ambassadorlabs
#3 Deploy an In-Cluster Edge Stack
• Pros
• Edge management is simplified into a single stack
• Supports cloud native best practices: “single
source of truth”, GitOps etc
• Cons
• Potentially a large architectural/responsibility shift
• Platform team must learn about new proxy
technologies and edge components
47

48. @danielbryantuk | @ambassadorlabs48
Wrapping Up

49. @danielbryantuk | @ambassadorlabs
In Conclusion
• Edge/API gateways have undergone a series of evolutions, driven by architecture and tech
• Hardware -> software
• Networking Layer 4 -> Layer 7
• Centralized management -> decentralised
• Adopting microservices/Kubernetes changes workflow
• Scale edge management
• Support multi-protocol and cross-functional requirements
• Choose your cloud API gateway intentionally (and also your platform components in general)
49

50. @danielbryantuk | @ambassadorlabs
app.getambassador.io
thenewstack.io/learning-kubernetes-the-need-for-a-realistic-playground

51. @danielbryantuk | @ambassadorlabs
Many thanks!
• Learn more:
• www.getambassador.io/learn/building-kubernetes-platform
• www.getambassador.io/podcasts
• www.infoq.com/profile/Daniel-Bryant
• Find me in:
• Datawire OSS Slack: d6e.co/slack
• Twitter @danielbryantuk
51