This document discusses moving a microservices architecture to the next level with service meshes. It introduces Istio as a service mesh for Kubernetes that provides traffic management, observability, and security capabilities. Istio uses the sidecar proxy pattern to enable features like mutual TLS, timeouts, retries, and circuit breakers to help solve challenges of microservices distribution like service discovery, load balancing, and failure handling. The document demonstrates Istio's capabilities and recommends adopting its features incrementally to gain experience before fully implementing a service mesh.

1. Moving your Microservices Architecture to
the next level with Service Meshes

2. 2
Ludovic Toison
Head Of Engineering
ludovic.toison@nfq.asia
xin chào!
Vietnam . Thailand . Egypt
2

3. 3
Microservices Architecture
From Monolith to Microservices

4. 4
Microservices Architecture
From Monolith to Microservices

5. 5
Microservices Architecture
Solving some interesting challenges
Design complexity
Data consistency
Testing & Debugging
Deploying
Monitoring
Operating
Securing

6. 6
Microservices Architecture
Solving some interesting challenges
Design complexity
Data consistency
Testing & Debugging
Deploying
Monitoring
Operating
Securing

7. 7
Microservices Architecture
Solving the distribution of your components
Service Discovery
Load-Balancing
Scalability
Failover

8. 8
Microservices Architecture
Solving the distribution of your components
Service Discovery
Load-Balancing
Scalability
Failover

9. 9
Microservices Architecture
Solving the distribution of your components
Service Discovery
Load-Balancing
Scalability
Failover

10. 10
Microservices Architecture
Solving the distribution of your components
Service Discovery
Load-Balancing
Scalability
Service
Failover

11. 11
Microservices Architecture
Solving the distribution of your components
Service Discovery
Load-Balancing
Scalability
Service
Failover
Replicas

12. 12
Microservices Architecture
Solving the distribution of your components
Service Discovery
Load-Balancing
Scalability
Service
Failover
Replicas
Controller

13. 13
What else?

14. 14
Microservices Architecture
Fallacies of distributed computing
Network is reliable
Latency is zero
Bandwidth is infinite
Network is secure
Source: https://nighthacks.com/jag/res/Fallacies.html

15. 15
Services Meshes
Infrastructure layer supporting inter-service communication
Service Mesh Service Mesh Service Mesh

16. 16
Istio
A Service-Mesh for Kubernetes
Service Mesh Service Mesh Service Mesh
Traffic Management Observability Security

17. 17
Istio
Sidecar pattern

18. 18
Istio
Mutual TLS – inter-service communication is encrypted & authenticated
Communication Mode
▪ Clear (strict)
▪ TLS (strict)
▪ Clear + TLS (mixed)
Encryption Scope
▪ Service
▪ Namespace
▪ Mesh Wide
Service Mesh Service Mesh
mTLS

19. 19
Istio
Timeout – handling latency
Service Mesh Service Mesh
✓ Delegate Timeout Management
to the Service Mesh instead of
the Application
✓ Combine Timeout with Retry
strategy

20. 20
Istio
Define the Timeout using a VirtualService (CRD)
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: backend
spec:
hosts:
- name: backend
http:
- timeout: 5s
route:
- destination:
host: backend

21. 21
Istio
Retry – Handling failures
Service Mesh Service Mesh
✓ Delegate Retry Management
to the Service Mesh instead of
the Application
✓ Compensate a failing instance

22. 22
Istio
Define the Retry using a VirtualService (CRD)
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: backend
spec:
hosts:
- name: backend
http:
- route:
- destination:
host: backend
retries:
attempt: 3
perTryTimeout: 2s
retryOn: 5xx

23. 23
Istio
Circuit Breaker – Backoff Pressure Management
Service Mesh Service Mesh
✓ Provide a quick response on
failure
✓ Limit network overhead in case
of a failure
✓ Isolate a failing component

24. 24
Istio
Circuit Breaker – Backoff Pressure Management
Source: https://martinfowler.com/bliki/CircuitBreaker.html

25. Istio
Define the Circuit Breaker using a DestinationRule (CRD)
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: backend
spec:
host: backend
trafficPolicy:
connectionPool:
tcp:
maxConnections: 100
http:
http1MaxPendingRequests: 1000
maxRequestsPerConnection: 10
outlierDetection:
consecutive5xxErrors: 6
interval: 5m
baseEjectionTime: 15m
maxEjectionPercent: 30 25
Restriction
Eviction

26. 26
DEMO

27. Istio
Observability using Kiali
27

28. 28
Istio
Distributed Tracing using Jaeger
28

29. 29
DEMO

30. Istio
Blue Green Deployment
30
✓ Validate a feature with Production traffic
✓ Qualify the feature (errors, performance,
dependencies)
✓ Ability to rollback anytime
✓ Limited impact in case of a failure
Service
Mesh
Service
Mesh
75%
25%
v1
v2

31. Istio
Canary Deployment
31
✓ Validate a feature with Production traffic
✓ Qualify the feature (errors, performance,
dependencies)
✓ Ability to rollback anytime
✓ Limited impact in case of a failure
Service
Mesh
Service
Mesh
location: ‘’*’’
location: ‘’fr’’
v1
v2

32. Istio
Traffic Mirroring (asynchronous)
32
Service
Mesh
Service
Mesh
100%
100%
✓ Validate a feature with Production traffic
✓ Qualify the feature (errors, performance,
dependencies)
✓ No impact in case of a failure
v1
v2

33. Istio
Conclusion
33
Service Mesh provides an agnostic solution for Microservices management
▪ Traffic Management
▪ Observability
▪ Security
Istio integrates well with Kubernetes but also complexify the setup
• Additional CRDs to manipulate
• Additional components to operate

34. Istio
Recommendations
34
Adopt an incubation approach to start with limited set of features
▪ mTLS
▪ Retry / Timeout / Circuit Breaker
▪ Observability
Master how features operate and adjust constantly
• Analyze (deep-dive)
• Measure (impact, benefits)
• Adapt (kill or extends)

35. Any questions?
Thank you!
35
Ludovic Toison
ludovic.toison@nfq.asia