1.
GARY ARORA
Cloud Solutions Architect
Deloitte Consulting LLP
What a Mesh!
Navigating a Mesh of Microservices in the
new Cloud-Native World with Istio
CLOUD EXPO 2018

2.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 2
Topics
Cloud Native
Containers
Microservices
Service Mesh
Istio

3.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 3
A Brief History of Computing

4.
Virtualization
• You likely need to buy new hardware every time you
need a new server
• Often longer downtimes due to outages. Slow DR
• Unit of measure: physical servers
Physical Servers are great but…
• Run multiple virtual machines on one physical server
• Cost savings through reduced footprint, faster server
provisioning, and improved disaster recovery (DR)
• Unit of measure: virtual machines
…consider Virtualization
Hardware
CPU Memory Disk Network
Operating System
Application
Hardware
CPU Memory Disk Network
OS 1
App 1 App 2 App 3
OS 2 OS 3
Hypervisor
Operating System
Virtualization
2001

5.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 5
Public Cloud
• Limited by finite resources for servers, storage,
network IOPS
• Limited by budget and talent pool of in-house IT Ops
for maintenance & high-availability
On Premise Data Centers are great but…
• Potentially limitless capacity for servers, storage,
network IOPS
• Cloud providers performs all maintenance and
guarantees availability via various SLAs
…consider moving to Cloud
Virtualization
2001
Public Cloud
2006
Data
Operating System
Virtualization
Storage
Networking
Hardware
Applications
Data
Operating System
Virtualization
Storage
Networking
Hardware
Applications
Customer’sResponsibility
Customer’sResponsibility

6.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 6
Simply being in the cloud is
no longer enough to remain
competitive!
92% of enterprises are already
using the public cloud in 2018
Source: State of Cloud Survey, Rightscale

7.
Cloud Native
An approach to maximizing the capabilities
of the cloud by rethinking technology
choices, architecture, and operations
CI/CD
DevOpsContainers
Microservices
Faster Time To
Market
Support rapid
Innovations
Increased
Resiliency
Agility & Scalability Increased Security Lower Costs
Benefits
Components

8.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 8
Containers
• VMs can take up a lot of system resources with full
virtual OS, RAM, and CPU cycles
• Startup time in minutes
• Limits the portability of applications
Virtual Machines are great but…
• Reduced IT management resources
• Startup time in milliseconds
• A portable, consistent operating environment for
development, testing, and deployment
…consider Containerization
Virtualization
2001
Public Cloud
2006
Containers
2013
Hardware
CPU Memory Disk Network
Guest
OS
App 1 App 2 App 3
Guest
OS
Guest
OS
Hypervisor
Host Operating System
Bins/Libs Bins/Libs Bins/Libs
Hardware
CPU Memory Disk Network
App 1 App 2 App 3
Docker Engine
Host Operating System
Bins/Libs Bins/Libs Bins/Libs

9.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 9
Container Management
• Containers cannot inherently communicate with
each other
• Containers have to be managed & deployed
appropriately
• Native auto scaling is not possible
• Distributed traffic is still challenging
Containers are great but…
• Automate packaging
• Service Discovery & Load
Balancing
• Storage Orchestration
• Self-Healing
…consider Container Management
• Batch Execution
• Secret & Configuration
Mgmt.
• Horizontal Scaling
• Automatic Rollbacks &
Rollouts
Virtualization
2001
Public Cloud
2006
Containers
2013
Container Mgmt.
2015

10.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 10
Microservices
2011-14
Microservices
• Scaling capabilities independently is challenging
• Changing one thing requires deploying everything
• Require extensive manual testing
Monoliths are great but…
• Independently develop and deploy services
• Organized around business capabilities
• Effective fault isolation
• Scalability and reusability
• Polygot
…consider Microservices
Database
Payment
Checkout
Shopping Cart
3rd Party Integrations
Recommendation
Product Catalog
Frontend
Shipping
Email Notifications
Users
Business Logic
Payment
Checkout Shopping Cart
3rd Party Integrations
RecommendationsProduct Catalog
Frontend
Shipping
Email Notifications
Users
Virtualization
2001
Public Cloud
2006
Containers
2013
Container Mgmt.
2015

11.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 11
It becomes an orders of magnitude larger problem to network and debug a mesh of microservices
When Microservices Grow…
Payment
Checkout
Shopping Cart
Recommendations
Product Catalog
Frontend
Shipping
Email Notifications
Reviews User Profile
Real time Pricing
3rd Party Price Match
Order Auto-
Replenishment
Shopping History
Refunds
Customer
Personalization
Customer Service
In-store inventory
Users
Promotions Marketing Emails
Payment
Shopping Cart
Recommendations
Shipping
Email Notifications
User Profile
Real time Pricing
3rd Party Price Match
Order Auto-
Replenishment
Shopping History
Refunds
Customer
Personalization
Customer Service
Marketing Campaigns
Premium User Services
The Eight Fallacies of
Distributed Computing
1. The network is reliable
2. Latency is zero
3. Bandwidth is infinite
4. The network is secure
5. Topology doesn't change
6. There is 1 administrator
7. Transport cost is zero
8. Network is homogeneous
Source: Peter Deutsch, 1994, Sun Microsystems

12.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 12
Global Microservices Trend
91% are using or have plans to use
microservices
92% expect to grow their use of
microservices in the coming year
86% expect microservices to be
the default within five years
Microservices have become mainstream… …but adoption still has many challenges
99% report challenges with using
microservices
73% find troubleshooting is harder in a
microservices environment
98% of those that face issues identifying
the root cause of issues in microservices
environments report it has a direct
business impact
91% 99%
92%
86%
73%
98%
Source: Online survey by Dimensional Research & LightStep | April 2018

13.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 13
Service Mesh
They still require a lot of management to:
• Debug network & infrastructure issues
• Create dependency graphs & latency
• Authenticate, rate limit, access control
Microservices are great but… …consider service mesh
• Tracing
• Monitoring
• Logging
• Authentication
• Pipeline
• Resilience
• Routing
• Discovery
Microservices
2011-14
Virtualization
2001
Public Cloud
2006
Containers
2013
Container Mgmt.
2015
Service Mesh.
2018

14.
Istio
An open services platform to manage service interactions across containers and VM-based workloads
“Kubernetes changed how we deploy applications.
Istio is going to change how we connect, manage, and secure them”
~Kelsey Hightower, Google Developer Advocate
Connect
Intelligently control the flow of
traffic and API calls between
services
Secure
Manages authentication,
authorization, and encryption
of communications
Control
Apply policies and ensure that
they are enforced, and that
resources are aptly distributed
Observe
Rich automated tracing,
monitoring, and logging for all
services
Source: Istio documentation

15.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 15
Istio Architecture
1. Envoy is a sidecar proxy
mediates all traffic
2. Mixer enforces access control
and collects telemetry data
3. Pilot provides service discovery,
& traffic management via rules
4. Citadel provides service-to-
service and user authentication
Source: Istio documentation

16.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 16
Istio Key Capabilities: Traffic Management
• Request routing
• Discovery and load balancing
• Handling failures
• Rate limiting, circuit breakers, A/B
testing
• Fault injection
• Rule configuration
Source: Istio documentation
Traffic splitting decouples from
infrastructure scaling
Content-based traffic steering

17.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 17
Traffic Management: Canary Deployments
• Once regularly used in coal
mining as an early detectors of
toxic gases
• Incremental rollouts to a subset
of users
• Can detect potential bugs and
disruption without affecting
every other system running.
With IstioWithout Istio
http://reviews.example.com
Prod 75% Canary 25%
Prod 90% Canary 10%
Prod 75%
Prod 90% Canary 10%
Canary 25%
http://reviews.example.com

18.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 18
Apply
Rules
25%
75%
reviews.example.com
http://reviews.example.com
Source: Istio documentation
Traffic & Routing
rules
Common scenarios where this is used include A/B testing or canary rollouts.
Traffic Management Example

19.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 19
Istio Key Capabilities: Security
• Key and certificate management
• Perimeter proxies
• Authentication
• Transport authentication
• Origin authentication
• Mutual TLS authentication
• Authorization
• Role-based Access Control (RBAC)
• Namespace-level
• Service-level
• Method-level access control
Source: Istio documentation

20.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 20
Istio Key Capabilities: Telemetry & Distributed Tracing
• Telemetry is automatically
injected in any service pod
providing Prometheus-style
network and L7 protocol metrics
• Istio dynamically traces the flow
and chained connections of the
microservices mesh.
Source: Istio documentation

21.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 21
Summary
Higher Abstraction. Increased focus on functionality
• Servers -> Virtual Machines -> Containers -> Serverless
• Monolithics -> Microservices
• Proprietary -> Open Source
• Single Vendor -> Cross-vendor

22.
Copyright © 2018 Deloitte Development LLC. All rights reserved. | 22
By 2020, 75% of application
purchases supporting digital
business will be “build” not
“buy”
Source: Gartner Forecast Analysis 2015

23.
About Deloitte
Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee (“DTTL”), its network of member firms, and
their related entities. DTTL and each of its member firms are legally separate and independent entities. DTTL (also referred to as “Deloitte Global”) does not
provide services to clients. In the United States, Deloitte refers to one or more of the US member firms of DTTL, their related entities that operate using the
“Deloitte” name in the United States and their respective affiliates. Certain services may not be available to attest clients under the rules and regulations of
public accounting. Please see www.deloitte.com/about to learn more about our global network of member firms.
This publication contains general information only and Deloitte is not, by means of this publication, rendering accounting, business, financial, investment, legal,
tax, or other professional advice or services. This publication is not a substitute for such professional advice or services, nor should it be used as a basis for
any decision or action that may affect your business. Before making any decision or taking any action that may affect your business, you should consult a
qualified professional advisor. Deloitte shall not be responsible for any loss sustained by any person who relies on this publication.
Copyright © 2018 Deloitte Development LLC. All rights reserved.
@AroraGary