SpringOne Platform 2017 Ramiro Salas, Pivotal The concept of a service mesh represents a paradigm shift on application connectivity for distributed systems, with wide implications for analytics, policy and extensibility. In this talk, we will explain what a service mesh is, the power it brings to microservices, and its impact on Cloud Foundry and K8s, both separately and together. We will also discuss the implications for the traditional network infrastructure, and the shifting of responsibilities from L3/4 to L7, and our current thinking of using Istio to integrate all abstractions.

1. Connecting All Abstractions with Istio
Ramiro Salas, Product Lead, Networking @ Pivotal
Laurent Demailly, Staff Engineer @ Google

2. It’s all about microservices

3. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Microservices expect the network to…
• Be available
• Be fast
• Be secure
• Have low latency
• Be transparent
• Never fail
3
Microservices
(but we all know better…)

4. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Network’s expectations for microservices
• Implement your own Service Discovery
• Implement your own app-level flow control
• Implement your own internal security
• Implement your own protection in case of network
partitioning
4
Networks

5. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
But Microservices have more complex needs
• Service health checks
• Extensive metrics
• Efficient patterns like Circuit Breakers
• Higher-level flow controls and QoS
• Advanced Policy controls
5
Microservices
And IP networks can’t really provide all that by themselves

6. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Microservices Networks
HTTP/2
gRPC
MQTT
CoAP
Protocols Start to Evolve
6
• Encryption everywhere
• HTTP(s)/1.1
• Raw TCP
• Raw UDP
• Multi-ports
Unfortunately, this is
Not enough!
But there is still need for:

7. 30 years ago, we would have created
entirely new network stacks to satisfy
these needs

8. But that’s not realistic
today…

9. So… we augment the network…

10. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
μS μS μS
μS μS μS
μS μS μS
So instead of a direct communication model
10

11. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
μS
μS
μS
μS
μS
μS
μS
μS
μS
SC
SC
SC
SC
SC
SC
SC
SC
SC
We insert sidecars alongside every μS
(your new data plane)
11
mTLS

12. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
μS
μS
μS
μS
μS
μS
μS
μS
μS
SC
SC
SC
SC
SC
SC
SC
SC
SC
Control Plane
And add a Control Plane of course…
12
mTLS
Control Plane

13. And that is essentially
a Service Mesh

14. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
The Market Today:
Data Planes:
Linkerd, Envoy, HAProxy, Traefik, more…
Control Planes:
• Istio, Synapse, Nelson
14
Some can work together

15. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Istio Overview
• Istio is Platform agnostic
• Already being progressively added to Cloud Foundry
• Backed by Google, IBM and Lyft
• Very lightweight Sidecar (Envoy)
• Dynamic, vibrant community
• Emphasis on pluggability and extensibility
• Enabled for distributed tracing
• Intelligent traffic management
• Distributed security
15

16. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Lyft’s Envoy
A C++ based L4/L7 proxy
Low memory footprint
Battle-tested @ Lyft
○ 100+ services
○ 10,000+ VMs
○ 2M req/s
16
• Features
• HTTP/2 & gRPC
• Zone-aware load balancing w/ failover
• Health checks, circuit breakers, timeouts,
retry budgets
• No hot reloads - API driven config updates
• Istio’s contributions:
• Transparent proxying w/
SO_ORIGINAL_DST
• Traffic routing and splitting
• Request tracing using Zipkin
• Fault injection

17. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
North-South, and East-West
• Outbound features:
• Service authentication
• Load balancing
• Retry and circuit breaker
• Fine-grained routing
• Telemetry
• Request Tracing
• Fault Injection
17
svcA
EnvoyEnvoy
Service A
svcB
Envoy
Service B
External
Services
HTTP/1.1, HTTP/2,
gRPC, TCP with or
without TLS
HTTP/1.1, HTTP/2,
gRPC, TCP with or
without TLS
Internet
• Inbound features:
• Service authentication
• Authorization
• Rate limits
• Load shedding
• Telemetry
• Request Tracing
• Fault Injection

18. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Control Plane Components
18
svcA
Envoy
Pod/AI
Service A
svcB
Envoy
Service B
Pilot
Control Plane API
Mixer
Control flow during
request processing Istio-Auth

19. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Mixer
• Policy evaluation and telemetry
• Precondition checking
• Quotas & Rate Limiting
• Primary point of extensibility
• Enabler for platform mobility
• Operator-focused configuration
model
19

20. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Pilot
• Lifecycle of Envoy instances
• API for Service Discovery
• Load Balancing Pools
• Routing Tables
2
0

21. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Istio-Auth
• Strong service-to-service and end-user authentication using mutual TLS
• Built-in identity and credential management
• RBAC and authorization hooks in roadmap
21

22. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Istio Environments
Istio 0.2 supports, today:
• Kubernetes single cluster, multiple namespaces
• Initial Consul, Eureka adapters
• Mesh expansion: Securely add VMs to your cluster
Already being worked on/soon:
• Kubernetes multi-cluster
• Non flat network hybrid
• And… CF !
2
2

23. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Istio Mesh Expansion
● Provisioning
● Networking (ILB)
● DNS
● Iptables
● Security: Istio Auth Node Agent
● Debian packages for Envoy, Agent...
23
Reviews-v1
Reviews-v2
Reviews-v3
Productpage
Details
Ratings
HTTP Requests
MySQL DB
Reviews namespace
Product namespace Product namespace
VMs (Onprem namespace)
Running in a VM, outside of K8s
VMs (Onprem namespace)

24. Istio on CF Container Runtime

25. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
CF Container Runtime (formerly Kubo)
Invented to delight a person who creates Kubernetes clusters
Vanilla K8s, and very up-to-date
All controlled and managed by BOSH
Strong isolation and multitenancy is achieved by hard separation
Istio, like K8s, doesn’t do strong multitenancy very well
So Istio on K8s + CFCR == goodness
2
5

26. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Setting it up
1. Create a K8s cluster:
./bin/deploy_k8s ~/kubo-env/kubo my-cluster
2. Access your K8s environment:
./bin/set_kubeconfig ~/kubo-env/kubo my-cluster
3. Get your Istio release:
curl -L https://git.io/getLatestIstio | sh -
4. Setup mTLS for all sidecars and automatic injection with Istio initializer:
kubectl apply -f install/kubernetes/istio-auth.yaml
kubectl apply -f
install/kubernetes/istio-initializer.yaml
2
6

27. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Verify Istio is installed and running
$ kubectl get svc -n istio-system
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-egress 10.83.247.89 <none> 80/TCP 5h
istio-ingress 10.83.245.171 35.184.245.62 80:32730/TCP,443:30574/TCP 5h
istio-pilot 10.83.251.173 <none> 8080/TCP,8081/TCP 5h
istio-mixer 10.83.244.253 <none> 9091/TCP,9094/TCP,42422/TCP 5h
27

28. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Deploy your app
$ kubectl create -f <your-app-spec>.yaml
OR
$ kubectl create -f <(istioctl kube-inject -f <your-app-spec>.yaml)
2
8
(Only needed if you didn’t install the Istio initializer)

29. Istio on CF Application Runtime

30. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Istio on CF Application Runtime
• Phase 1: Envoy on App Instances for mTLS
• Part of the route integrity work.
• Phase 2: Ingress
• Envoy for ingress
• Istio Pilot providing route updates
• Removal of NATS and GoRouters
• Phase 3: East-West traffic
• App-to-app and app-to-service via Envoy w/mTLS
• Integration with policy engine. Mixer exploration.
30

31. Integrating all abstractions via Istio

32. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Virtual Landscapes
Imagine a future where developers will be able to:
• cf push apps and microservices to CF Spaces, or functions to PFS.
• Seamlessly discover and consume other apps and functions in the same platform,
or different PCF foundations and clouds altogether.
• Bind against BOSH-provided services anywhere, or services running on on-
demand PKS-generated K8s clusters.
• Pods in K8s consuming apps and microservices in PAS or functions in PFS.
• All with mTLS, advanced telemetry, QoS, circuit breakers and ultra-granular end-
to-end policy controls.
32

33. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
A note of caution...
“Difficult to see. Always in motion the future is”

34. Thanks