API Days 2018 balance your latency budget

Balancing your latency budget
Ori Pekelman
CPO @platformsh
oripekelman everywhere github/linked-in/twitter

A model of latency
will make you
money.
What is a latency budget?

Latency is
non-productive
time.

SYSTEM DOING
SOMETHING
SYSTEM WAITING ON
SOME OTHER SYSTEM
DOING SOMETHING
THAT MIGHT BE
USEFUL
USER WAITING FOR
RESPONSE FROM
SYSTEM
API CALL

SYSTEM DOING
SOMETHING
SYSTEM WAITING ON
SOME OTHER SYSTEM
DOING SOMETHING
THAT MIGHT BE
USEFUL
USER WAITING FOR
RESPONSE FROM
SYSTEM
Facts of life?
API CALL

SYSTEM DOING
SOMETHING
SYSTEM WAITING ON
SOME OTHER SYSTEM
DOING SOMETHING
THAT MIGHT BE
USEFUL
USER WAITING FOR
RESPONSE FROM
SYSTEM
Grbll, but
maybe have to
live with it?
API CALL

SYSTEM DOING
SOMETHING
SYSTEM WAITING ON
SOME OTHER SYSTEM
DOING SOMETHING
THAT MIGHT BE
USEFUL
USER WAITING FOR
RESPONSE FROM
SYSTEM
Grbll, but
physics?
API CALL

Whatever your
system does that is
actually useful is
not latency.

You can not beat
physics. But you
can take it into
account.

1. The speed of light is 299,792 km/s
2. Around 200,000 km/s in fiber
3. Places on earth can be 20000 km apart
4. Fibers don’t actually go in straight lines
5. TCP requires a roundtrip.
6. TLS Handshake requires 2 extra roundtrips.
7. Paris is 9000 kms from San Francisco
Fighting with physics

Paris is at 130ms
TTFB from San
Francisco.
What is a latency budget?Fighting with physics

Non-Negotiable.

Unless … ?

What is a latency budget?FLAME GRAPHS ARE VISUALLY
PLEASING

What is a latency budget?AND THE EQUIVALENT OF GOING
DOWN A RABBIT HOLE

AND THE EQUIVALENT OF GOING
DOWN A RABBIT HOLE
This
.. is
this
And
this
… is
this

FLAME GRAPHS HELP US
OPTIMIZE WHAT IS SLOW
BECAUSE LATENCY IS A
RUSSIAN DOLL

System boundary (LB)
API CALL
Edge
API
Network
Latency
Network
Latency
Network
Latency
Network
Latency
Very small
Network
Latency
SYSTEM DOING
SOMETHING
USEFUL
The user waits this

1. Physics
2. Robustness (remember the “very small
network latency”)?
Stuff that goes in a latency budget

I. Writes are more valuable than reads.
II. Some writes change the world and the world
is slow.
POSTULATES

If your API makes you get food it better be slow
because fast-food is bad for you.
What is a latency budget?A WORLD CHANGING WRITE

In our case probably the most valuable write
would be:
$ curl -X POST
https://api.platform.sh/projects/jeh
oi1h/environments/staging/merge -H
'authorization: Bearer 60...fg1d'

Which means put staging into production, go
live!
$ curl -X POST

Which means put staging into production, go
live! (On Friday).
$ curl -X POST

System boundary (LB)
API CALL
Edge
API
Network
Latency
Network
Latency
Network
Latency
Network
Latency
Very small
Network
Latency
SYSTEM DOING
SOMETHING
SLOW
The user waits this

We are fast. But deploying a new cluster can
take anywhere between 30 seconds and a couple
of minutes.
A backup can take a few minutes up until it is
safely stored remotely.
LATENCY TOLERANCE IS
RELATIVE TO VALUE

SOME DATA MUST STAY COHESIVE
1. Because the co-localisation of data and
processing is an invariant. Not going away.
Because Physics.
2. Because compliance; Data-localisation is
important;
3. Because git is already distributed and it
makes no sense, no sense at all to distribute
the authority that will control your
deployment
ONE MORE POSTULATE

What is a latency budget?YOUR GIT CONTAINER IS GOING
TO BE SOMEWHERE
SYSTEM DOING
SOMETHING
SLOW, FAR AWAY
But your developers may be everywhere… so how do
we beat physics?

-H 'authorization: Bearer
60...fg1d'
What is a latency budget?POLITESSE OBLIGE: SAYING NO
SHOULD BE FAST
Remember that bit from our very valuable API call
Verifying a token is something you can do at the
edge. Saying No can be done close to the client. (You
still need to distribute secrets, so non-trivial).

SHOULD BE FAST
Remember that bit about trading a bit of latency for
robustness?
Adding yet another HTTP server, really close to the
client adds a probably unnoticeable amount of
latency in the worst case scenario, but already allows
you to say NO, really fast.
Also incredibly fast TTFB. 4ms is nicer than 130ms.

SHOULD BE FAST
Your edge can be a full-blown API gateway. Some
Lambda/Functions. Or a CDN. You will find a bunch
of vendors around that will be more than happy to
extol the virtues of theirs...

Your API does not deliver value equally across
the board.
In our very useful API method we can already
say no quickly.
But saying yes is also something that we
accelerate through decoupling.
What is a latency budget?Why model latency?

Interestingly enough `git merge` in our world
means:
● The world is in State A. It should be in State B.
● Please make the necessary arrangements.
A slow answer from our API would be “200 OK”
A better answer is “202 Accepted”.
What is a latency budget?YES IS QUICK WHEN YES IS MAYBE

System
boundary
(LB)
API CALL
Edge
API
Network
Latency
Network
Latency
Very small
Network
Latency
The user waits this

Euphemistically 202 is “non-commital”. But that
was a damn commit we sent there...
What is a latency budget?BUT HOW WOULD I KNOW IF
EVRERYTHING WENT FINE?
$ curl
oi1h/environments/master/subscribe

System
boundary
(LB)
API CALL
Edge
API
Network
Latency
Network
Latency
Very small
Network
Latency
The user waits no longer

We can subscribe to the answer before we start
the request …
We just beat physics.
What is a latency budget?BUT HOW WOULD I KNOW IF
EVRERYTHING WENT FINE?

Latency is part of
user-centric API
Design

Project to the edge.
Use event streams
for responses.
Slow is valuable.

And BTW for those that were here
for the previous talk… In
Platform.sh a Single API Call gives
you a fully consistent backup of all
of your microservices and their
backends. Even if some say physics
say we can’t.

API Days 2018 balance your latency budget

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Empfohlen

Empfohlen (20)

API Days 2018 balance your latency budget