Ingress solutions for publising microservices based applications on Kubernetes and Swarm

2. ● Javier Ramírez Urea
− Modern Infrastructures
− Dockerized at HoplaSoftware
− DCA, DCI, MSCE
jramirez@hoplasoftware.com
infra-team@hoplasoftware.com
frjaraur
hopla-training

3. Publishing Applications
“... Not a long time ago, there were
tons of Monoliths ...”
From Monoliths to Services Mesh
3

4. Publishing Applications
“... Evolution always brings new
problems to solve ...”
From Monoliths to Services Mesh
4

5. Publishing Applications
From Monoliths to Services Mesh
Monolith Application
5

6. Publishing Applications
From Monoliths to Services Mesh
Monolith Application Router Mesh
6

7. Publishing Applications
From Monoliths to Services Mesh
Monolith Application Router Mesh
GATEWAY
7

8. Publishing Applications
From Monoliths to Services Mesh
Monolith Application Router Mesh Service Mesh
GATEWAY
8

9. Publishing Applications
From Monoliths to Services Mesh
Monolith Application Router Mesh Service Mesh
GATEWAY
GATEWAY
9

10. Publishing Applications
From Monoliths to Services Mesh
Monolith Application Router Mesh Service Mesh
GATEWAY
GATEWAY
G
ATEW
AY
GATEWAY
GATEW
AY
GATEWAY
10

11. Publishing Applications
From Monoliths to Services Mesh
Monolith Application Router Mesh Service Mesh
GATEWAY
11

12. Publishing Applications
From Monoliths to Services Mesh
Router Mesh
GATEWAY
Solutions for publishing applications
must managed dynamic service
components routing when they
change:
● Manage North-South Traffic
● Resilience
● Scaling
● Host Maintenances
● Updatings/Rollbacks
...Etc…..
12

13. Publishing Applications
From Monoliths to Services Mesh
Router Mesh
GATEWAY
Solutions for publishing applications
must managed dynamic service
components routing when they
change:
● Manage North-South Traffic
● Resilience
● Scaling
● Host Maintenances
● Updatings/Rollbacks
...Etc…..
13

14. Publishing Applications
• Kubernetes publishing
• Pods are mortal and their number can be scale up/down
whenever we need it.
• Services are abstraction which defines a logical set of Pods
and a policy by which to access them:
■ Endpoints API updated whenever the set of Pods in a
Service changes, for Kubernetes-native applications.
■ A Virtual-IP-based bridge to Services which redirects to
the backend Pods, for non-native applications.
Containers Routing Publishing Mesh
14

15. Publishing Applications
• Cluster IP
• Node Port
• Load Balancer
• External Name
• External IPs
Kubernetes Service Types
15

16. Publishing Applications
SERVICE
KUBE-PROXY
POD POD POD
Cluster IP
16

17. Publishing Applications
SERVICE
KUBE-PROXY
POD POD POD
Cluster IP
Just for Debugging
or allowing internal
traffic, displaying
internal
dashboards, etc.
17

18. Publishing Applications
SERVICE
POD POD POD
LOAD BALANCER
VIP
Load Balancer
18

19. Publishing Applications
SERVICE
POD POD POD
LOAD BALANCER
VIP
Load Balancer
The big downside is that each
service you expose with a
LoadBalancer will get its own IP
address, and you have to pay for a
LoadBalancer per exposed service,
which can get expensive!
19

20. Publishing Applications
SERVICE
POD POD POD
Service
Port
Service
Port
Service
Port
EXTERNAL LOAD BALANCER
NodePort
20

21. Publishing Applications
SERVICE
POD POD POD
Service
Port
Service
Port
Service
Port
EXTERNAL LOAD BALANCER
NodePort
There are many downsides to this
method:
● You can only have once service
per port
● You can only use ports
30000–32767 (configurable, but
fixed)
● If your Node/VM IP address
change, you need to deal with that
21

22. Publishing Applications
Ingress Controllers
SERVICE_APP1
POD_APP1 POD_APP1 POD_APP1
Ingress
Port
INGRESS CONTROLLER
SERVICE_APP2
POD_APP2 POD_APP2
app1.example.com
myapps.example.com/app1
app2.example.com
myapps.example.com/app2
EXTERNAL LOAD BALANCER
22

23. Publishing Applications
Ingress Resource
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target:
/
spec:
rules:
- host: a.example.com
http:
paths:
- path: /testpath
backend:
serviceName: test
servicePort: 80
23

24. Publishing Applications
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target:
/
spec:
rules:
- host: a.example.com
http:
paths:
- path: /testpath
backend:
serviceName: test
servicePort: 80
Ingress Resource
24

25. Publishing Applications
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target:
/
spec:
rules:
- host: a.example.com
http:
paths:
- path: /testpath
backend:
serviceName: test
servicePort: 80
Ingress Resource
25

26. Publishing Applications
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target:
/
spec:
rules:
- host: a.example.com
http:
paths:
- path: /testpath
backend:
serviceName: test
servicePort: 80
Ingress Resource
26

27. Publishing Applications
Smart Routers
● Kubernetes Nginx
● Nginx/Nginx Plus
● Traëfik
Kubernetes Ingress Controllers Types
Router Controllers
● Cluster External Ingress Elements
Interaction:
○ Big IP F5 Ingress
○ Octavia Ingress Controller
○ GLBC (GCE L7 lbctl)
○ Netscaler
● Cluster Internal API Gateways:
○ Kong
Framework Specific
● Docker EE Interlock
● Openshift Router
27

28. Publishing Applications
Docker EE Layer 7 routing in UCP supports:
• High availability:
• Automatic configuration
• Scalability
• TLS
• Context-based routing
• Host mode networking
• Security
• Allows Redirects
• Persistent Sessions
• Websockets
• Canary Deployments with merge
Docker EE Interlock
28

29. Publishing Applications
SERVICE_APP1
POD_APP1 POD_APP1 POD_APP1
Ingress
Port
INTERLOCK
SERVICE_APP2
POD_APP2 POD_APP2
app1.example.com
myapps.example.com/app1
app2.example.com
myapps.example.com/app2
EXTERNAL LOAD BALANCER
ucp-interlock ucp-interlock-extension ucp-interlock-proxy ucp-interlock-proxy
Docker EE Interlock
29

30. Publishing Applications
Main Features
● Based on Community Nginx with third-party modules
● Supported by Kubernetes community
● Host rules (allow merging)
● HTTP load balancing extensions (Annotations and ConfigMap)
● HTTP/S, TCP/UDP and TCP SSL Passthrough
● Websocket
● Prometheus Integration
● Dynamic reconfiguration with third-party LUA plugin
Kubernetes Nginx Ingress Controller
30

31. Publishing Applications
Main Features
● HTTP/S host header and path routing
● Circuit Breakers
● Status Based Health Checks
● Routing with Basic authentication
● Extended configuration via toml
● Routing Priorities
Traëfik Ingress Controller
31

32. Publishing Applications
NGINX Ingress Controller
32

33. Publishing Applications
NGINX and NGINX Plus supports the following Ingress features:
● SSL termination
● Path-based rules
● Multiple host names and Mergeable Ingress resources
● Websocket, which allows you to load balance Websocket applications.
● SSL Services, which allows you to load balance HTTPS applications.
● Rewrites, which allows you to rewrite the URI of a request before
sending it to the application.
● Customized templates using annotations
NGINX Ingress Controller
33

34. Publishing Applications
Nginx Plus Only Available Ingress features:
● Dynamic Reconfiguration without reloading
● Complex Health Checks with pattern matching
● Extended Nginx Plus Dashboard
● Session Persistence.
● Support for JWTs to authenticate requests by validating JSON Web
Tokens (JWTs).
NGINX Plus Ingress Controller
34

35. Publishing Applications
NGINX Ingress Controller
35

36. Publishing Applications
Kong Ingress Controller
36

37. Publishing Applications
Main features:
● Act as Controller for Kong (almost everything Kong does it is available).
● Kong ingress controller does not use Services to route traffic to the
pods. Instead it uses the Endpoints API to bypass kube-proxy to allow
Kong features like session affinity and custom load balancing
algorithms. It also removes overhead, such as conntrack entries for
iptables DNAT.
● Add New KongPlugin Resource
● Routing through API gateway
Kong Ingress Controller
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: add-ratelimiting-to-route
namespace: kong
config:
hour: "100"
limit_by: "ip"
second: "10"
37

38. Publishing Applications
38

39. Publishing Applications
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: blue|red-app
spec:
replicas: 2
selector:
matchLabels:
app: blue|red
template:
metadata:
labels:
app: blue|red
spec:
containers:
- name: blue|red
image: codegazers/colors:1.5
env:
- name: COLOR
value: blue|red
ports:
- containerPort: 3000
apiVersion: v1
kind: Service
metadata:
name: blue|red-svc
spec:
ports:
- port: 80
targetPort: 3000
protocol: TCP
name: http
selector:
app: blue|red
Colors Deployment
https://raw.githubusercontent.com/Codegazers/k8s-vagrant/master/demo/colors.yml
39

40. Publishing Applications
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: colors-ingress
spec:
rules:
- host: blue.example.com
http:
paths:
- backend:
serviceName: blue-svc
servicePort: 80
- host: red.example.com
http:
paths:
- backend:
serviceName: red-svc
servicePort: 80
Colors Ingress Resource
https://raw.githubusercontent.com/Codegazers/k8s-vagrant/master/demo/colors-ingress.yml
40

41. Publishing Applications
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: colors-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.com/health-checks: "true"
spec:
rules:
- host: blue.example.com
http:
paths:
- backend:
serviceName: blue-svc
servicePort: 80
- host: red.example.com
http:
paths:
- backend:
serviceName: red-svc
servicePort: 80
Ingress Resource with Health Checks
https://raw.githubusercontent.com/Codegazers/k8s-vagrant/master/demo/colors-with-health.yml
spec:
containers:
- name: red
image: codegazers/colors:1.5
ports:
- containerPort: 3000
env:
- name: COLOR
value: "red"
readinessProbe:
httpGet:
port: 3000
path: /health
httpHeaders:
- name: AppStatus
value: "UP"
initialDelaySeconds: 1
periodSeconds: 5
timeoutSeconds: 4
successThreshold: 2
failureThreshold: 3
Deployment Checks
41

42. Publishing Applications
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: colors-ingress
annotations:
nginx.org/rewrites: "serviceName=red-svc rewrite=/;"
spec:
rules:
- host: blue.example.com
http:
paths:
- path: /red/
backend:
serviceName: red-svc
servicePort: 80
- backend:
serviceName: blue-svc
servicePort: 80
- host: red.example.com
http:
paths:
- backend:
serviceName: red-svc
servicePort: 80
Ingress Controller Rewriting
https://raw.githubusercontent.com/Codegazers/k8s-vagrant/master/demo/colors-ingress-rewrite.yml
42

43. Publishing Applications
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: colors-ingress
annotations:
nginx.com/sticky-cookie-services: "serviceName=red-svc red_svc_id
expires=60s path=/;"
spec:
rules:
- host: blue.example.com
http:
paths:
- path: /red/
backend:
serviceName: red-svc
servicePort: 80
- backend:
serviceName: blue-svc
servicePort: 80
- host: red.example.com
http:
paths:
- backend:
serviceName: red-svc
servicePort: 80
Ingress Controller Session Persistence
https://raw.githubusercontent.com/Codegazers/k8s-vagrant/master/demo/colors-ingress-sticky.yml
43

44. Publishing Applications
Ingress Controller Rate Limiting
https://raw.githubusercontent.com/Codegazers/k8s-vagrant/master/demo/colors-ingress-sticky.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: colors-ingress
annotations:
"rate-limiting.plugin.konghq.com":"add-ratelimiting-to-route"
spec:
rules:
- host: blue.example.com
http:
paths:
- backend:
serviceName: blue-svc
servicePort: 80
- host: red.example.com
http:
paths:
- backend:
serviceName: red-svc
servicePort: 80
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: add-ratelimiting-to-route
config:
minute: 20
limit_by: ip
second: 5
Kong Plugin Resource
44

45. Publishing Applications
Docker EE2 Interlock
https://raw.githubusercontent.com/Codegazers/k8s-vagrant/master/demo/colors-interlock.yml
version: "3.3"
# A Docker Compose file for configuration of the development environment
services:
red:
image: codegazers/colors:1.5
environment:
- COLOR=red
deploy:
replicas: 3
labels:
com.docker.lb.hosts: red.example.com
com.docker.lb.network: colors
com.docker.lb.port: 3000
networks:
- colors
blue:
image: codegazers/colors:1.5
environment:
- COLOR=blue
deploy:
replicas: 2
labels:
com.docker.lb.hosts: blue.example.com
com.docker.lb.network: colors
com.docker.lb.port: 3000
networks:
- colors
networks:
colors:
driver: overlay
45

46. Publishing Applications
Docker EE2 Interlock
https://raw.githubusercontent.com/Codegazers/k8s-vagrant/master/demo/colors-interlock.yml
version: "3.3"
# A Docker Compose file for configuration of the development environment
services:
red:
image: codegazers/colors:1.5
environment:
- COLOR=red
deploy:
replicas: 3
labels:
com.docker.lb.hosts: red.example.com
com.docker.lb.network: colors
com.docker.lb.port: 3000
networks:
- colors
blue:
image: codegazers/colors:1.5
environment:
- COLOR=blue
deploy:
replicas: 2
labels:
com.docker.lb.hosts: blue.example.com
com.docker.lb.network: colors
com.docker.lb.port: 3000
networks:
- colors
networks:
colors:
driver: overlay
46

47. jramirez@hoplasoftware.com
infra-team@hoplasoftware.com
frjaraur
hopla-training