How we built out a multi-tenant solution on AWS's hosted Kubernetes (EKS)

1. Thursday the 27th of June, 2019
Multi-Tenant*
Platform on EKS
info@container-solutions.com
container-solutions.com
Ian Crosby

2. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Agenda
■ Kubernetes Background
■ Introduce Project
■ Architecture
■ Multi Tenancy
■ AWS Integrations
■ Benefits
■ Challenges
■ Life Changing Conclusion

3. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

4. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Fast / Customer-Centric
SafeandReliable
LIMITS SPEED,
SAFETY AND
COMPLEXITY
ACHIEVE
QUALITY
THROUGH
SPEED

5. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Microservices

6. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Containers

7. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Virtual Machine vs Container

8. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Orchestration

9. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Container Orchestrators
■ Docker Swarm from Docker
■ Mesos from Apache
■ DC/OS from Mesosphere
■ Nomad from Hashicorp
■ CCP from Cisco
■ OpenShift by Red Hat
■ Fleet by CoreOS (Deprecated)
■ ...

10. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

11. Kubernetes

12. Kubernetes
● Built by Google (Based on Borg)
● Donated to CNCF
● Fully Open Source
● Healthy and Robust Community and Ecosystem

13. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

14. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Flavors of Kubernetes
Self Hosted
■ Kubespray
■ Kubeadm
■ K3s
Managed
■ EKS (Amazon)
■ GKE (Google)
■ AKS (Microsoft)
■ IKS (IBM)
PaaS
■ OpenShift
■ Digital Ocean
■ Giant Swarm

15. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

16. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Mandate: Migration to EKS
■ Online Platform for Factories
■ Software as a Service
■ Small/medium applications
■ Low traffic volume
■ Lots of Clients (and customization)
■ Complete Environment per Customer
■ Entirely AWS Based

17. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
vs.
■ Proven Solution
■ Strong Integration
■ Uncertain Future
■ Vendor Specific
■ Cloud Agnostic
■ More Features
■ Supporting Tooling
■ Cost (?)

18. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Migration Path
■ Containerize Applications
■ Define Kubernetes Configuration
■ Integrate with AWS Services
■ Incremental Migration
■ Invisible to Dev Team

19. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
New Architecture

20. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Multitenancy

21. ... architecture in which a single instance
of software runs on a server and serves
multiple tenants ... designed to provide
every tenant a dedicated share of the
instance - including its data, configuration

22. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Why do we want Multitenancy?
■ Efficiency
■ Simplicity
■ Cost

23. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
The Risks of Multitenancy
■ Security
■ “Noisy Neighbor”
■ Single Point of Failure

24. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Multi
Instance
Multi
Tenant

25. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Multitenancy on Kubernetes
Leverage native Kubernetes Features
■ Namespaces
■ Network Policies
■ Resource Limits
■ RBAC
■ Ingress

26. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
YAML

27. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
EKS Cluster

28. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

29. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
EKS Cluster
Namespace: Client A Namespace: Client B

30. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

31. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
EKS Cluster
Namespace A Namespace B
Worker Worker

32. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
EKS Cluster
Namespace A Namespace B
Worker Worker

33. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

34. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Kubernetes Cluster
Namespace A Namespace B
Network Policy
Worker Worker

35. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
EKS Cluster
Namespace A Namespace B
Network Policy
Worker
Worker
Worker
Worker
Worker
Worker
Worker Worker

36. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

37. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
EKS Cluster
Limit RangeLimit Range
Namespace A Namespace B
Network Policy
Worker
Worker
Worker
Worker
Worker
Worker
Worker Worker

38. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
EKS Cluster
Resource Limits
Namespace: Client A
Resource Limits
Namespace: Client B
Network Policy
Client A
Ingress
Controller
Worker
Worker

39. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

40. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
EKS Cluster
Resource Limits
Namespace: Client A
Resource Limits
Namespace: Client B
Network Policy
Client A
Ingress
Controller
Worker
Worker

41. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Kubernetes Cluster
Resource LimitsResource Limits
Namespace A Namespace B
App 1 App 2
Resource LimitsResource Limits
Namespace C Namespace D
App 3 App 4
Resource LimitsResource Limits
Namespace E Namespace F
App 5 App 6
Resource LimitsResource Limits
Namespace G Namespace H
App 7 App 8

42. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Multitenant-ish

43. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

44. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
AWS Integrations
ALB Ingress Controller
■ Dynamically spins up Application Load Balancer
■ Built-in Health Checks
■ TLS / Cert Management
■ Subnet and VPN integration

45. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
https://aws.amazon.com/blogs/op
ensource/kubernetes-ingress-aw
s-alb-ingress-controller/

46. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
AWS Integrations
External DNS
■ Integrates with Route53
■ Dynamically control DNS records

47. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

48. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
AWS Integrations
CloudWatch
■ Integration with Kubernetes Cluster
■ Application and Platform Metrics
■ Not ideal (yet)

49. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

50. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
AWS Integrations
Databases (RDS and Elasticache)
■ Don’t manage if you don’t have to
■ Create an abstraction layer
Amazon
Elasticache
Amazon
RDS

51. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Benefits
■ Lower Costs
■ Simpler Setup
■ Easier Client Onboarding
■ Poised for Growth
■ (Less) Vendor Lock-In

52. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Challenges
■ Load Balancing / Ingress
■ Tuning Resource Requests/Limits
■ Application Updates
■ Cluster Size
■ EKS new-ness

53. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Was it all Worth it?

54. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby

55. container-solutions.com info@container-solutions.com Multi-Tenancy with Kubernetes @IanDCrosby
Was it Worth it?
■ Large investment with little *visible* benefit
■ Building for Scale
■ Cost savings + Potential
■ Ready for Further Improvements

56. Thank You
Questions?
Thanks to unsplash.com for all the pics