SlideShare ist ein Scribd-Unternehmen logo
1 von 156
Downloaden Sie, um offline zu lesen
Konrad `@ktosopl` Malawski
need for async
hot pursuit
for Scalable apps
Konrad `ktoso` Malawski
Akka Team,
Reactive Streams TCK
(we’re renaming soon!)
Konrad `@ktosopl` Malawski
akka.io
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl
lambdakrk.pl
(we’re renaming soon!)
Nice to meet you!
Who are you guys?
High Performance Software Development
For the majority of the time,

high performance software development 

is not about compiler hacks and bit twiddling. 



It is about fundamental design principles that are 

key to doing any effective software development.
Martin Thompson
practicalperformanceanalyst.com/2015/02/17/getting-to-know-martin-thompson-...
Agenda
• Why?
• Async and Synch basics / definitions
• Async where it matters: Scheduling
• How NOT to measure Latency
• Concurrent < lock-free < wait-free
• I/O: IO, AIO, NIO, Zero
• C10K: select, poll, epoll / kqueue
• Distributed Systems: Where Async is at Home
• Wrapping up and Q/A
Why?
Why this talk?
“The free lunch is over”
Herb Sutter
(A Fundamental Turn Toward Concurrency in Software)
Why this talk?
“The free lunch is over”
Herb Sutter
(A Fundamental Turn Toward Concurrency in Software)
How and why
Asynchronous Processing
comes into play for
Scalability and Performance.
Why this talk?
“The free lunch is over”
Herb Sutter
(A Fundamental Turn Toward Concurrency in Software)
How and why
Asynchronous Processing
comes into play for
Scalability and Performance.
I want you guys make
well informed decisions,
not buzzword driven development.
Sync / Async Basics
Sync / Async
Sync / Async
Sync / Async
Sync / Async
Sync / Async
Sync / Async
Sync / Async
Sync / Async
Sync / Async
Highly parallel systems
Event loops
Actors
Async where it matters:
Async where it matters: Scheduling
Async where it matters: Scheduling
Async where it matters: Scheduling
Async where it matters: Scheduling
Async where it matters: Scheduling
Async where it matters: Scheduling
Async where it matters: Scheduling
Async where it matters: Scheduling
Async where it matters: Scheduling
Scheduling (notice they grey sync call)
Scheduling (notice they grey sync call)
Scheduling (now with Async db call)
Scheduling (now with Async db call)
Scheduling (now with Async db call)
Scheduling (now with Async db call)
Lesson learned:
Blocking is indistinguishable from being very slow.
Latency
Latency Quiz #1: Which one is Latency?
By the queueing theory definitions:
Latency Quiz #1: Which one is Latency?
By the queueing theory definitions:
Latency Quiz #1: Which one is Latency?
By the queueing theory definitions:
Latency Quiz #1: Which one is Latency?
By the queueing theory definitions:
Latency Quiz #1: Which one is Latency?
Latency Quiz #1: Which one is Latency?
Sometimes devs use Latency and Response Time as the same thing.
That’s OK, as long as both sides know which one they are talking about.
Latency Quiz #2
Gil Tene style, see:“How NOT to Measure Latency”
Is 10s latency acceptable in your app?
Is 200ms latency acceptable?
How about most responses within 200ms?
So mostly 20ms and some 1 minute latencies is OK?
Do people die when we go above 200ms?
So 90% below 200ms, 99% bellow 1s, 99.99% below 2s?
Latency in the “real world”
Gil Tene style, see:“How NOT to Measure Latency”
“Our response time is 200ms average,
stddev is around 60ms”
— a typical quote
Latency in the “real world”
Gil Tene style, see:“How NOT to Measure Latency”
“Our response time is 200ms average,
stddev is around 60ms”
— a typical quote
Latency does NOT behave like normal distribution!
“So yeah, our 99,99%’ is…”
http://hdrhistogram.github.io/HdrHistogram/
Hiccups
Gil Tene style, see:“How NOT to Measure Latency”
Hiccups
Gil Tene style, see:“How NOT to Measure Latency”
Hiccups
Gil Tene style, see:“How NOT to Measure Latency”
Hiccups
Gil Tene style, see:“How NOT to Measure Latency”
Hiccups
Gil Tene style, see:“How NOT to Measure Latency”
Hiccups
Lesson learned:
Use precise language when talking about latencies.
Measure the right thing!
“Trust no-one, bench everything!”
Verify the results!
Ask on groups or forums.
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
“concurrent” data structure
Concurrent < lock-free < wait-free
What can happen in concurrent data structures:
A tries to write; B tries to write; B wins!
A tries to write; C tries to write; C wins!
A tries to write; D tries to write; D wins!
A tries to write; B tries to write; B wins!
A tries to write; E tries to write; E wins!
A tries to write; F tries to write; F wins!
…
Moral?
1) Thread A is a complete loser.
2) Thread A may never make progress.
What can happen in concurrent data structures:
A tries to write; B tries to write; B wins!
C tries to write; C wins!
D tries to write; D wins!
B tries to write; B wins!
E tries to write; E wins!
F tries to write; F wins!
…
Moral?
1) Thread A is a complete loser.
2) Thread A may never make progress.
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-less
Remember: Concurrency is NOT Parallelism.
Rob Pike - Concurrency is NOT Parallelism (video)
def offer(a: A): Boolean // returns on failure


def add(a: A): Unit // throws on failure

def put(a: A): Boolean // blocks until able to enqueue
Concurrent < lock-free < wait-free
concurrent data structure
<
lock-free* data structure
* lock-free a.k.a. lockless
What lock-free programming looks like:
An algorithm is lock-free if it satisfies that:
When the program threads are run sufficiently long,
at least one of the threads makes progress.
Concurrent < lock-free < wait-free
* Both versions are used: lock-free / lockless
class CASBackedQueue[A] {
val _queue = new AtomicReference(Vector[A]())
// does not block, may spin though
@tailrec final def put(a: A): Unit = {
val queue = _queue.get
val appended = queue :+ a
if (!_queue.compareAndSet(queue, appended))
put(a)
}
}
Concurrent < lock-free < wait-free
class CASBackedQueue[A] {
val _queue = new AtomicReference(Vector[A]())
// does not block, may spin though
@tailrec final def put(a: A): Unit = {
val queue = _queue.get
val appended = queue :+ a
if (!_queue.compareAndSet(queue, appended))
put(a)
}
}
Concurrent < lock-free < wait-free
class CASBackedQueue[A] {
val _queue = new AtomicReference(Vector[A]())
// does not block, may spin though
@tailrec final def put(a: A): Unit = {
val queue = _queue.get
val appended = queue :+ a
if (!_queue.compareAndSet(queue, appended))
put(a)
}
}
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
Concurrent < lock-free < wait-free
“concurrent” data structure
<
lock-free* data structure
<
wait-free data structure
* Both versions are used: lock-free / lockless
Concurrent < lock-free < wait-free
Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms

Maged M. Michael Michael L. Scott
An algorithm is wait-free if every operation has a bound on
the number of steps the algorithm will take before the
operation completes.
wait-free: j.u.c.ConcurrentLinkedQueue
Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms

Maged M. Michael Michael L. Scott
public boolean offer(E e) {
checkNotNull(e);
final Node<E> newNode = new Node<E>(e);
for (Node<E> t = tail, p = t;;) {
Node<E> q = p.next;
if (q == null) {
// p is last node
if (p.casNext(null, newNode)) {
// Successful CAS is the linearization point
// for e to become an element of this queue,
// and for newNode to become "live".
if (p != t) // hop two nodes at a time
casTail(t, newNode); // Failure is OK.
return true;
}
// Lost CAS race to another thread; re-read next
}
else if (p == q)
// We have fallen off list. If tail is unchanged, it
// will also be off-list, in which case we need to
// jump to head, from which all live nodes are always
// reachable. Else the new tail is a better bet.
p = (t != (t = tail)) ? t : head;
else
// Check for tail updates after two hops.
p = (p != t && t != (t = tail)) ? t : q;
}
}
This is a modification of the Michael & Scott algorithm,
adapted for a garbage-collected environment, with support
for interior node deletion (to support remove(Object)).


For explanation, read the paper.
Lesson learned:
Different ways to construct concurrent data structures.
Pick the right impl for the right job.
I / O
IO / AIO
IO / AIO / NIO
IO / AIO / NIO / Zero
Synchronous I / O
— Havoc Pennington
(HAL, GNOME, GConf, D-BUS, now Typesafe)
When I learned J2EE about 2008 with some of my
desktop colleagues our reactions included something like:



”wtf is this sync IO crap, 

where is the main loop?!” :-)
Interruption!
CPU: User Mode / Kernel Mode
Kernels and CPUs
Kernels and CPUs
Kernels and CPUs
Kernels and CPUs
Kernels and CPUs
http://wiki.osdev.org/Context_Switching
Kernels and CPUs
[…] switching from user-level to kernel-level 

on a (2.8 GHz) P4 is 1348 cycles. 



[…] Counting actual time, the P4 takes 481ns […]
http://wiki.osdev.org/Context_Switching
I / O
I / O
I / O
I / O
I / O
I / O
“Don’t worry.
It only gets worse!”
I / O
“Don’t worry.
It only gets worse!”
Same data in 3 buffers!4 mode switches!
Asynchronous I / O [Linux]
Linux AIO = JVM NIO
Asynchronous I / O [Linux]
NewIO… since 2004!
(No-one calls it “new” any more)
Linux AIO = JVM NIO
Asynchronous I / O [Linux]
Less time wasted waiting.
Same amount of buffer copies.
ZeroCopy = sendfile [Linux]
“Work smarter.
Not harder.”
http://fourhourworkweek.com/
ZeroCopy = sendfile [Linux]
ZeroCopy = sendfile [Linux]
Data never leaves kernel mode!
ZeroCopy…
Lesson learned:
By picking the right tool,
you can avoid wasting CPU time.
Asynchronous IO will not beat a synchronous seek
throughput-wise but it can allow higher scalability.
C10K and beyond
C10K and beyond
“10.000 concurrent connections”
Not a new problem, pretty old actually: ~12 years old.
http://www.kegel.com/c10k.html
C10K and beyond- why?
It’s not about performance.
It’s about scalability.
These are orthogonal things.
Threading differences: apache / nginx
select/poll
C10K – poll
C10K – poll
C10K – poll
epoll
C10K – epoll [Linux]
C10K – epoll [Linux]
C10K – epoll [Linux]
C10K – epoll [Linux]
C10K – epoll [Linux]
C10K
O(n) is a no-go for epic scalability.
C10K
O(n) is a no-go for epic scalability.
State of Linux scheduling:
O(n) O(1) CFS (O(1)/ O(log n))
And Socket selection:
Select/Poll O(n) EPoll (O(1))
C10K
O(n) is a no-go for epic scalability.
State of Linux scheduling:
O(n) O(1) CFS (O(1))
And Socket selection:
Select/Poll O(n) EPoll (O(1))
O(1) IS a go for epic scalability.
Moral:
Distributed Systems
Distributed Systems
“… in which the failure of a computer
you didn't even know existed can
render your own computer unusable.”
— Leslie Lamport
http://research.microsoft.com/en-us/um/people/lamport/pubs/distributed-system.txt
Distributed Systems
The bigger the system,
the more “random” latency / failure noise.
Embrace instead of hiding it.
Distributed Systems
Backup Requests
Backup requests
A technique for fighting “long tail latencies”.
By issuing duplicated work, when SLA seems in danger.
Backup requests
Backup requests
Backup requests - send
Backup requests - send
Backup requests - send
Backup requests
Avg Std dev 95%ile 99%ile 99.9%ile
No backups 33 ms 1524 ms 24 ms 52 ms 994 ms
After 10ms 14 ms 4 ms 20 ms 23 ms 50 ms
After 50ms 16 ms 12 ms 57 ms 63 ms 68 ms
Jeff Dean - Achieving Rapid Response Times in Large Online Services
Peter Bailis - Doing Redundant Work to Speed Up Distributed Queries
Akka - Krzysztof Janosz @ Akkathon, Kraków - TailChoppingRouter (docs, pr)
Lesson learned:
Backup requests allow
trade-off increased load
for decreased latency.
Distributed Systems
Combined Requests
&
Back-pressure
Combined requests & back-pressure
Combined requests & back-pressure
No no no…!
Not THAT Back-pressure!
Combined requests & back-pressure
THAT kind of back-pressure:
www.reactive-streams.org
Combined requests & back-pressure
THAT kind of back-pressure:
www.reactive-streams.org
Combined requests & back-pressure
THAT kind of back-pressure:
www.reactive-streams.org
Combined requests & back-pressure
THAT kind of back-pressure:
www.reactive-streams.org
Combined requests
A technique for avoiding duplicated work.
By aggregating requests, possibly increasing latency.
“Wat?Why would I increase latency!?”
Combined requests
Combined requests
Combined requests
Combined requests
Lesson learned:
Back-pressure saves systems from overload.
Combined requests trade higher latency,
for less work for the downstream.
Wrapping up
Wrapping up
You don’t need to be a great mechanic
to be a great racing driver,
but you must understand how your bolid works!
~ Martin Thompson (in context of Mechanical Sympathy)
Wrapping up
• Someone has to bite the bullet though!
• We’re all running on real hardware.
• Libraries do it so you don’t have to - pick the right one!
Be aware that:
Wrapping up
• Keep your apps pure
• Be aware of internals
• Async all the things!
• Messaging all the way!
• Someone has to bite the bullet though!
• We’re all running on real hardware.
• Libraries do it so you don’t have to - pick the right one!
Be aware that:
Links
• akka.io
• reactive-streams.org
• akka-user

• Gil Tene - How NOT to measure latency, 2013
• Jeff Dean @Velocity 2014
• Alan Bateman, Jeanfrancois Arcand (Sun) Async IO Tips @ JavaOne
• http://linux.die.net/man/2/select
• http://linux.die.net/man/2/poll
• http://linux.die.net/man/4/epoll
• giltene/jHiccup
• Linux Journal: ZeroCopy I, Dragan Stancevis 2013
• Last slide car picture: http://actu-moteurs.com/sprint/gt-tour/jean-
philippe-belloc-un-beau-challenge-avec-le-akka-asp-team/2000
Links
• http://wiki.osdev.org/Context_Switching
• CppCon: Herb Sutter "Lock-Free Programming (or, Juggling Razor Blades)"
• http://www.infoq.com/presentations/reactive-services-scale
• Gil Tene’s HdrHistogram.org
• http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
• Rob Pike - Concurrency is NOT Parallelism (video)
• Brendan Gregg - Systems Performance: Enterprise and the Cloud (book)
• http://psy-lob-saw.blogspot.com/2015/02/hdrhistogram-better-latency-capture.html
• Jeff Dean, Luiz Andre Barroso - The Tail at Scale (whitepaper,ACM)
• http://highscalability.com/blog/2012/3/12/google-taming-the-long-latency-tail-when-
more-machines-equal.html
• http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system-
architects/
• Marcus Lagergren - Oracle JRockit:The Definitive Guide (book)
• http://mechanical-sympathy.blogspot.com/2013/08/lock-based-vs-lock-free-
concurrent.html
• Handling of Asynchronous Events - http://www.win.tue.nl/~aeb/linux/lk/lk-12.html
• http://www.kegel.com/c10k.html
Links
• www.reactivemanifesto.org/
• Seriously the only right way to micro benchmark on the JVM:
• JMH openjdk.java.net/projects/code-tools/jmh/
• JMH for Scala: https://github.com/ktoso/sbt-jmh
• http://www.ibm.com/developerworks/library/l-async/
• http://lse.sourceforge.net/io/aio.html
• https://code.google.com/p/kernel/wiki/AIOUserGuide
• ShmooCon: C10M - Defending the Internet At Scale (Robert Graham)
• http://blog.erratasec.com/2013/02/scalability-its-question-that-drives-us.html#.VO6E11PF8SM
•User-level threads....... with threads. - Paul Turner @ Linux Plumbers Conf 2013
•https://www.facebook.com/themainstreetpiggyboys/photos/a.
1390784047896753.1073741829.1390594984582326/1423592294615928/?
type=1&theater for the Rollin’ Cuy on last slide
Special thanks to:
• Aleksey Shipilëv
• Andrzej Grzesik
• Gil Tene
• Kirk Pepperdine
• Łukasz Dubiel
• Marcus Lagergren
• Martin Thompson
• Mateusz Dymczyk
• Nitsan Wakart
Thanks guys, you’re awesome.
alphabetically, mostly
• Peter Lawrey
• Richard Warburton
• Roland Kuhn
• Sergey Kuksenko
• Steve Poole
• Viktor Klang a.k.a. √
• Antoine de Saint Exupéry;-)
• and the entire AkkaTeam
• the Mechanical Sympathy Mailing List
Learn more at:
• SCKRK.com –
• Java.pl – Polish (Kraków) JUG
• KrakowScala.pl – Kraków Scala User Group
• LambdaKRK.pl – Lambda Lounge Kraków
• GeeCON.org – awesome “all around the JVM” conference, 

as always: Kraków May 2015
• GeeCON Tricity (at the beach!) - September 2015
• GeeCON Prague - October 2015
Software Craftsmanship Kraków
Computer Science Whitepaper Reading Club Kraków
Thanks & Q/A!
ktoso @ typesafe.com
twitter: ktosopl
github: ktoso
team blog: letitcrash.com
home: akka.io
©Typesafe 2015 – All Rights Reserved

Weitere ähnliche Inhalte

Was ist angesagt?

Creating_scheduled_reports_with_Zabbix.pdf
Creating_scheduled_reports_with_Zabbix.pdfCreating_scheduled_reports_with_Zabbix.pdf
Creating_scheduled_reports_with_Zabbix.pdf
ITDispendik
 
[社内共有会]AWS NAT-GW導入と構成変化 2年運用して 同時接続数 秒間100->10万へ成長
[社内共有会]AWS NAT-GW導入と構成変化 2年運用して 同時接続数 秒間100->10万へ成長[社内共有会]AWS NAT-GW導入と構成変化 2年運用して 同時接続数 秒間100->10万へ成長
[社内共有会]AWS NAT-GW導入と構成変化 2年運用して 同時接続数 秒間100->10万へ成長
Takahiro Moteki
 

Was ist angesagt? (20)

L2延伸を利用したクラウド移行とクラウド活用術
L2延伸を利用したクラウド移行とクラウド活用術L2延伸を利用したクラウド移行とクラウド活用術
L2延伸を利用したクラウド移行とクラウド活用術
 
Amazon Aurora Deep Dive (db tech showcase 2016)
Amazon Aurora Deep Dive (db tech showcase 2016)Amazon Aurora Deep Dive (db tech showcase 2016)
Amazon Aurora Deep Dive (db tech showcase 2016)
 
Cognitoハンズオン
CognitoハンズオンCognitoハンズオン
Cognitoハンズオン
 
Oracleからamazon auroraへの移行にむけて
Oracleからamazon auroraへの移行にむけてOracleからamazon auroraへの移行にむけて
Oracleからamazon auroraへの移行にむけて
 
【NGINXセミナー】API ゲートウェイとしてのNGINX Plus活用方法
【NGINXセミナー】API ゲートウェイとしてのNGINX Plus活用方法【NGINXセミナー】API ゲートウェイとしてのNGINX Plus活用方法
【NGINXセミナー】API ゲートウェイとしてのNGINX Plus活用方法
 
AWS運用管理のベストプラクティス hinemosクラウド管理オプションのご紹介
AWS運用管理のベストプラクティス hinemosクラウド管理オプションのご紹介AWS運用管理のベストプラクティス hinemosクラウド管理オプションのご紹介
AWS運用管理のベストプラクティス hinemosクラウド管理オプションのご紹介
 
20180220 AWS Black Belt Online Seminar - Amazon Container Services
20180220 AWS Black Belt Online Seminar - Amazon Container Services20180220 AWS Black Belt Online Seminar - Amazon Container Services
20180220 AWS Black Belt Online Seminar - Amazon Container Services
 
Zabbixによるオートスケーリングクラスタ監視とオペレーション自動化
Zabbixによるオートスケーリングクラスタ監視とオペレーション自動化Zabbixによるオートスケーリングクラスタ監視とオペレーション自動化
Zabbixによるオートスケーリングクラスタ監視とオペレーション自動化
 
Creating_scheduled_reports_with_Zabbix.pdf
Creating_scheduled_reports_with_Zabbix.pdfCreating_scheduled_reports_with_Zabbix.pdf
Creating_scheduled_reports_with_Zabbix.pdf
 
MySQLとPostgreSQLと日本語全文検索 - Azure DatabaseでMroonga・PGroongaを使いたいですよね!?
MySQLとPostgreSQLと日本語全文検索 - Azure DatabaseでMroonga・PGroongaを使いたいですよね!?MySQLとPostgreSQLと日本語全文検索 - Azure DatabaseでMroonga・PGroongaを使いたいですよね!?
MySQLとPostgreSQLと日本語全文検索 - Azure DatabaseでMroonga・PGroongaを使いたいですよね!?
 
[Cloud OnAir] Google Cloud とつなぐ色々な方法 〜 つなぐ方法をゼロからご紹介します〜 2019年1月31日 放送
[Cloud OnAir] Google Cloud とつなぐ色々な方法 〜 つなぐ方法をゼロからご紹介します〜 2019年1月31日 放送[Cloud OnAir] Google Cloud とつなぐ色々な方法 〜 つなぐ方法をゼロからご紹介します〜 2019年1月31日 放送
[Cloud OnAir] Google Cloud とつなぐ色々な方法 〜 つなぐ方法をゼロからご紹介します〜 2019年1月31日 放送
 
Db2 Warehouse on Cloud Flex ご紹介資料 2020年3月版
Db2 Warehouse on Cloud Flex ご紹介資料 2020年3月版Db2 Warehouse on Cloud Flex ご紹介資料 2020年3月版
Db2 Warehouse on Cloud Flex ご紹介資料 2020年3月版
 
PostgreSQLモニタリングの基本とNTTデータが追加したモニタリング新機能(Open Source Conference 2021 Online F...
PostgreSQLモニタリングの基本とNTTデータが追加したモニタリング新機能(Open Source Conference 2021 Online F...PostgreSQLモニタリングの基本とNTTデータが追加したモニタリング新機能(Open Source Conference 2021 Online F...
PostgreSQLモニタリングの基本とNTTデータが追加したモニタリング新機能(Open Source Conference 2021 Online F...
 
AWS Black Belt Online Seminar AWS 認定クラウドプラクティショナー取得に向けて
AWS Black Belt Online Seminar AWS 認定クラウドプラクティショナー取得に向けてAWS Black Belt Online Seminar AWS 認定クラウドプラクティショナー取得に向けて
AWS Black Belt Online Seminar AWS 認定クラウドプラクティショナー取得に向けて
 
AWS Blackbelt 2015シリーズ Amazon Storage Service (S3)
AWS Blackbelt 2015シリーズ Amazon Storage Service (S3)AWS Blackbelt 2015シリーズ Amazon Storage Service (S3)
AWS Blackbelt 2015シリーズ Amazon Storage Service (S3)
 
Amazon Aurora
Amazon AuroraAmazon Aurora
Amazon Aurora
 
[社内共有会]AWS NAT-GW導入と構成変化 2年運用して 同時接続数 秒間100->10万へ成長
[社内共有会]AWS NAT-GW導入と構成変化 2年運用して 同時接続数 秒間100->10万へ成長[社内共有会]AWS NAT-GW導入と構成変化 2年運用して 同時接続数 秒間100->10万へ成長
[社内共有会]AWS NAT-GW導入と構成変化 2年運用して 同時接続数 秒間100->10万へ成長
 
Migrate from Oracle to Aurora PostgreSQL: Best Practices, Design Patterns, & ...
Migrate from Oracle to Aurora PostgreSQL: Best Practices, Design Patterns, & ...Migrate from Oracle to Aurora PostgreSQL: Best Practices, Design Patterns, & ...
Migrate from Oracle to Aurora PostgreSQL: Best Practices, Design Patterns, & ...
 
Postgres connections at scale
Postgres connections at scalePostgres connections at scale
Postgres connections at scale
 
カジュアルにVPC作った結果がこれだよ!
カジュアルにVPC作った結果がこれだよ!カジュアルにVPC作った結果がこれだよ!
カジュアルにVPC作った結果がこれだよ!
 

Andere mochten auch

Ebay legacy-code-retreat
Ebay legacy-code-retreatEbay legacy-code-retreat
Ebay legacy-code-retreat
Konrad Malawski
 
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka StreamsFresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Konrad Malawski
 

Andere mochten auch (20)

DDDing Tools = Akka Persistence
DDDing Tools = Akka PersistenceDDDing Tools = Akka Persistence
DDDing Tools = Akka Persistence
 
Rethink Frontend Development With Elm
Rethink Frontend Development With ElmRethink Frontend Development With Elm
Rethink Frontend Development With Elm
 
JavaOne 2013: Java 8 - The Good Parts
JavaOne 2013: Java 8 - The Good PartsJavaOne 2013: Java 8 - The Good Parts
JavaOne 2013: Java 8 - The Good Parts
 
Open soucerers - jak zacząć swoją przygodę z open source
Open soucerers - jak zacząć swoją przygodę z open sourceOpen soucerers - jak zacząć swoją przygodę z open source
Open soucerers - jak zacząć swoją przygodę z open source
 
HBase RowKey design for Akka Persistence
HBase RowKey design for Akka PersistenceHBase RowKey design for Akka Persistence
HBase RowKey design for Akka Persistence
 
Scalding - the not-so-basics @ ScalaDays 2014
Scalding - the not-so-basics @ ScalaDays 2014Scalding - the not-so-basics @ ScalaDays 2014
Scalding - the not-so-basics @ ScalaDays 2014
 
Ebay legacy-code-retreat
Ebay legacy-code-retreatEbay legacy-code-retreat
Ebay legacy-code-retreat
 
TDD drogą do oświecenia w Scali
TDD drogą do oświecenia w ScaliTDD drogą do oświecenia w Scali
TDD drogą do oświecenia w Scali
 
Scala dsls-dissecting-and-implementing-rogue
Scala dsls-dissecting-and-implementing-rogueScala dsls-dissecting-and-implementing-rogue
Scala dsls-dissecting-and-implementing-rogue
 
Android at-xsolve
Android at-xsolveAndroid at-xsolve
Android at-xsolve
 
Git tak po prostu (SFI version)
Git tak po prostu (SFI version)Git tak po prostu (SFI version)
Git tak po prostu (SFI version)
 
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
 
Android my Scala @ JFokus 2013
Android my Scala @ JFokus 2013Android my Scala @ JFokus 2013
Android my Scala @ JFokus 2013
 
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
 
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka StreamsFresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
 
Disrupt 2 Grow - Devoxx 2013
Disrupt 2 Grow - Devoxx 2013Disrupt 2 Grow - Devoxx 2013
Disrupt 2 Grow - Devoxx 2013
 
The things we don't see – stories of Software, Scala and Akka
The things we don't see – stories of Software, Scala and AkkaThe things we don't see – stories of Software, Scala and Akka
The things we don't see – stories of Software, Scala and Akka
 
KrakDroid: Scala on Android
KrakDroid: Scala on AndroidKrakDroid: Scala on Android
KrakDroid: Scala on Android
 
Scalding - Hadoop Word Count in LESS than 70 lines of code
Scalding - Hadoop Word Count in LESS than 70 lines of codeScalding - Hadoop Word Count in LESS than 70 lines of code
Scalding - Hadoop Word Count in LESS than 70 lines of code
 
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...
 

Ähnlich wie Need for Async: Hot pursuit for scalable applications

Keynote: Building and Operating A Serverless Streaming Runtime for Apache Bea...
Keynote: Building and Operating A Serverless Streaming Runtime for Apache Bea...Keynote: Building and Operating A Serverless Streaming Runtime for Apache Bea...
Keynote: Building and Operating A Serverless Streaming Runtime for Apache Bea...
Flink Forward
 

Ähnlich wie Need for Async: Hot pursuit for scalable applications (20)

The Need for Async @ ScalaWorld
The Need for Async @ ScalaWorldThe Need for Async @ ScalaWorld
The Need for Async @ ScalaWorld
 
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...
 
Async programming and python
Async programming and pythonAsync programming and python
Async programming and python
 
Task parallel library presentation
Task parallel library presentationTask parallel library presentation
Task parallel library presentation
 
Defcon 22-paul-mcmillan-attacking-the-iot-using-timing-attac
Defcon 22-paul-mcmillan-attacking-the-iot-using-timing-attacDefcon 22-paul-mcmillan-attacking-the-iot-using-timing-attac
Defcon 22-paul-mcmillan-attacking-the-iot-using-timing-attac
 
PyCon Canada 2019 - Introduction to Asynchronous Programming
PyCon Canada 2019 - Introduction to Asynchronous ProgrammingPyCon Canada 2019 - Introduction to Asynchronous Programming
PyCon Canada 2019 - Introduction to Asynchronous Programming
 
Here comes the Loom - Ya!vaConf.pdf
Here comes the Loom - Ya!vaConf.pdfHere comes the Loom - Ya!vaConf.pdf
Here comes the Loom - Ya!vaConf.pdf
 
Building Hermetic Systems (without Docker)
Building Hermetic Systems (without Docker)Building Hermetic Systems (without Docker)
Building Hermetic Systems (without Docker)
 
Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014
 
Parallel Programming: Beyond the Critical Section
Parallel Programming: Beyond the Critical SectionParallel Programming: Beyond the Critical Section
Parallel Programming: Beyond the Critical Section
 
Asynchronous Python A Gentle Introduction
Asynchronous Python A Gentle IntroductionAsynchronous Python A Gentle Introduction
Asynchronous Python A Gentle Introduction
 
Node.js: CAMTA Presentation
Node.js: CAMTA PresentationNode.js: CAMTA Presentation
Node.js: CAMTA Presentation
 
Rust's Journey to Async/await
Rust's Journey to Async/awaitRust's Journey to Async/await
Rust's Journey to Async/await
 
Streaming 101: Hello World
Streaming 101:  Hello WorldStreaming 101:  Hello World
Streaming 101: Hello World
 
Highly concurrent yet natural programming
Highly concurrent yet natural programmingHighly concurrent yet natural programming
Highly concurrent yet natural programming
 
MERIMeeting du 27 mai 2014 - Parallel Programming
MERIMeeting du 27 mai 2014 - Parallel ProgrammingMERIMeeting du 27 mai 2014 - Parallel Programming
MERIMeeting du 27 mai 2014 - Parallel Programming
 
Beyond Breakpoints: A Tour of Dynamic Analysis
Beyond Breakpoints: A Tour of Dynamic AnalysisBeyond Breakpoints: A Tour of Dynamic Analysis
Beyond Breakpoints: A Tour of Dynamic Analysis
 
C# Async/Await Explained
C# Async/Await ExplainedC# Async/Await Explained
C# Async/Await Explained
 
Transactional Memory
Transactional MemoryTransactional Memory
Transactional Memory
 
Keynote: Building and Operating A Serverless Streaming Runtime for Apache Bea...
Keynote: Building and Operating A Serverless Streaming Runtime for Apache Bea...Keynote: Building and Operating A Serverless Streaming Runtime for Apache Bea...
Keynote: Building and Operating A Serverless Streaming Runtime for Apache Bea...
 

Mehr von Konrad Malawski

Reactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsReactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka Streams
Konrad Malawski
 

Mehr von Konrad Malawski (20)

Networks and Types - the Future of Akka @ ScalaDays NYC 2018
Networks and Types - the Future of Akka @ ScalaDays NYC 2018Networks and Types - the Future of Akka @ ScalaDays NYC 2018
Networks and Types - the Future of Akka @ ScalaDays NYC 2018
 
Akka Typed (quick talk) - JFokus 2018
Akka Typed (quick talk) - JFokus 2018Akka Typed (quick talk) - JFokus 2018
Akka Typed (quick talk) - JFokus 2018
 
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in't
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in'tScalaSwarm 2017 Keynote: Tough this be madness yet theres method in't
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in't
 
State of Akka 2017 - The best is yet to come
State of Akka 2017 - The best is yet to comeState of Akka 2017 - The best is yet to come
State of Akka 2017 - The best is yet to come
 
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYCBuilding a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
 
Akka-chan's Survival Guide for the Streaming World
Akka-chan's Survival Guide for the Streaming WorldAkka-chan's Survival Guide for the Streaming World
Akka-chan's Survival Guide for the Streaming World
 
Reactive integrations with Akka Streams
Reactive integrations with Akka StreamsReactive integrations with Akka Streams
Reactive integrations with Akka Streams
 
Not Only Streams for Akademia JLabs
Not Only Streams for Akademia JLabsNot Only Streams for Akademia JLabs
Not Only Streams for Akademia JLabs
 
Reactive Streams, j.u.concurrent & Beyond!
Reactive Streams, j.u.concurrent & Beyond!Reactive Streams, j.u.concurrent & Beyond!
Reactive Streams, j.u.concurrent & Beyond!
 
End to End Akka Streams / Reactive Streams - from Business to Socket
End to End Akka Streams / Reactive Streams - from Business to SocketEnd to End Akka Streams / Reactive Streams - from Business to Socket
End to End Akka Streams / Reactive Streams - from Business to Socket
 
The Cloud-natives are RESTless @ JavaOne
The Cloud-natives are RESTless @ JavaOneThe Cloud-natives are RESTless @ JavaOne
The Cloud-natives are RESTless @ JavaOne
 
Akka Streams in Action @ ScalaDays Berlin 2016
Akka Streams in Action @ ScalaDays Berlin 2016Akka Streams in Action @ ScalaDays Berlin 2016
Akka Streams in Action @ ScalaDays Berlin 2016
 
Krakow communities @ 2016
Krakow communities @ 2016Krakow communities @ 2016
Krakow communities @ 2016
 
Zen of Akka
Zen of AkkaZen of Akka
Zen of Akka
 
How Reactive Streams & Akka Streams change the JVM Ecosystem
How Reactive Streams & Akka Streams change the JVM EcosystemHow Reactive Streams & Akka Streams change the JVM Ecosystem
How Reactive Streams & Akka Streams change the JVM Ecosystem
 
Reactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsReactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka Streams
 
Reactive Streams / Akka Streams - GeeCON Prague 2014
Reactive Streams / Akka Streams - GeeCON Prague 2014Reactive Streams / Akka Streams - GeeCON Prague 2014
Reactive Streams / Akka Streams - GeeCON Prague 2014
 
2014 akka-streams-tokyo-japanese
2014 akka-streams-tokyo-japanese2014 akka-streams-tokyo-japanese
2014 akka-streams-tokyo-japanese
 
Distributed Consensus A.K.A. "What do we eat for lunch?"
Distributed Consensus A.K.A. "What do we eat for lunch?"Distributed Consensus A.K.A. "What do we eat for lunch?"
Distributed Consensus A.K.A. "What do we eat for lunch?"
 
Akka persistence == event sourcing in 30 minutes
Akka persistence == event sourcing in 30 minutesAkka persistence == event sourcing in 30 minutes
Akka persistence == event sourcing in 30 minutes
 

Kürzlich hochgeladen

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 

Kürzlich hochgeladen (20)

Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 

Need for Async: Hot pursuit for scalable applications

  • 1. Konrad `@ktosopl` Malawski need for async hot pursuit for Scalable apps
  • 2. Konrad `ktoso` Malawski Akka Team, Reactive Streams TCK (we’re renaming soon!)
  • 3. Konrad `@ktosopl` Malawski akka.io typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl lambdakrk.pl (we’re renaming soon!)
  • 4. Nice to meet you! Who are you guys?
  • 5. High Performance Software Development For the majority of the time,
 high performance software development 
 is not about compiler hacks and bit twiddling. 
 
 It is about fundamental design principles that are 
 key to doing any effective software development. Martin Thompson practicalperformanceanalyst.com/2015/02/17/getting-to-know-martin-thompson-...
  • 6. Agenda • Why? • Async and Synch basics / definitions • Async where it matters: Scheduling • How NOT to measure Latency • Concurrent < lock-free < wait-free • I/O: IO, AIO, NIO, Zero • C10K: select, poll, epoll / kqueue • Distributed Systems: Where Async is at Home • Wrapping up and Q/A
  • 8. Why this talk? “The free lunch is over” Herb Sutter (A Fundamental Turn Toward Concurrency in Software)
  • 9. Why this talk? “The free lunch is over” Herb Sutter (A Fundamental Turn Toward Concurrency in Software) How and why Asynchronous Processing comes into play for Scalability and Performance.
  • 10. Why this talk? “The free lunch is over” Herb Sutter (A Fundamental Turn Toward Concurrency in Software) How and why Asynchronous Processing comes into play for Scalability and Performance. I want you guys make well informed decisions, not buzzword driven development.
  • 11. Sync / Async Basics
  • 21. Highly parallel systems Event loops Actors Async where it matters:
  • 22. Async where it matters: Scheduling
  • 23. Async where it matters: Scheduling
  • 24. Async where it matters: Scheduling
  • 25. Async where it matters: Scheduling
  • 26. Async where it matters: Scheduling
  • 27. Async where it matters: Scheduling
  • 28. Async where it matters: Scheduling
  • 29. Async where it matters: Scheduling
  • 30. Async where it matters: Scheduling
  • 31. Scheduling (notice they grey sync call)
  • 32. Scheduling (notice they grey sync call)
  • 33. Scheduling (now with Async db call)
  • 34. Scheduling (now with Async db call)
  • 35. Scheduling (now with Async db call)
  • 36. Scheduling (now with Async db call)
  • 37. Lesson learned: Blocking is indistinguishable from being very slow.
  • 39. Latency Quiz #1: Which one is Latency? By the queueing theory definitions:
  • 40. Latency Quiz #1: Which one is Latency? By the queueing theory definitions:
  • 41. Latency Quiz #1: Which one is Latency? By the queueing theory definitions:
  • 42. Latency Quiz #1: Which one is Latency? By the queueing theory definitions:
  • 43. Latency Quiz #1: Which one is Latency?
  • 44. Latency Quiz #1: Which one is Latency? Sometimes devs use Latency and Response Time as the same thing. That’s OK, as long as both sides know which one they are talking about.
  • 45. Latency Quiz #2 Gil Tene style, see:“How NOT to Measure Latency” Is 10s latency acceptable in your app? Is 200ms latency acceptable? How about most responses within 200ms? So mostly 20ms and some 1 minute latencies is OK? Do people die when we go above 200ms? So 90% below 200ms, 99% bellow 1s, 99.99% below 2s?
  • 46. Latency in the “real world” Gil Tene style, see:“How NOT to Measure Latency” “Our response time is 200ms average, stddev is around 60ms” — a typical quote
  • 47. Latency in the “real world” Gil Tene style, see:“How NOT to Measure Latency” “Our response time is 200ms average, stddev is around 60ms” — a typical quote Latency does NOT behave like normal distribution! “So yeah, our 99,99%’ is…”
  • 49. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
  • 50. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
  • 51. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
  • 52. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
  • 53. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
  • 54. Lesson learned: Use precise language when talking about latencies. Measure the right thing! “Trust no-one, bench everything!” Verify the results! Ask on groups or forums.
  • 55. Concurrent < lock-free < wait-free
  • 56. Concurrent < lock-free < wait-free
  • 57. Concurrent < lock-free < wait-free
  • 58. Concurrent < lock-free < wait-free “concurrent” data structure
  • 59. Concurrent < lock-free < wait-free What can happen in concurrent data structures: A tries to write; B tries to write; B wins! A tries to write; C tries to write; C wins! A tries to write; D tries to write; D wins! A tries to write; B tries to write; B wins! A tries to write; E tries to write; E wins! A tries to write; F tries to write; F wins! … Moral? 1) Thread A is a complete loser. 2) Thread A may never make progress.
  • 60. What can happen in concurrent data structures: A tries to write; B tries to write; B wins! C tries to write; C wins! D tries to write; D wins! B tries to write; B wins! E tries to write; E wins! F tries to write; F wins! … Moral? 1) Thread A is a complete loser. 2) Thread A may never make progress. Concurrent < lock-free < wait-free
  • 61. Concurrent < lock-free < wait-less Remember: Concurrency is NOT Parallelism. Rob Pike - Concurrency is NOT Parallelism (video) def offer(a: A): Boolean // returns on failure 
 def add(a: A): Unit // throws on failure
 def put(a: A): Boolean // blocks until able to enqueue
  • 62. Concurrent < lock-free < wait-free concurrent data structure < lock-free* data structure * lock-free a.k.a. lockless
  • 64. An algorithm is lock-free if it satisfies that: When the program threads are run sufficiently long, at least one of the threads makes progress. Concurrent < lock-free < wait-free
  • 65. * Both versions are used: lock-free / lockless class CASBackedQueue[A] { val _queue = new AtomicReference(Vector[A]()) // does not block, may spin though @tailrec final def put(a: A): Unit = { val queue = _queue.get val appended = queue :+ a if (!_queue.compareAndSet(queue, appended)) put(a) } } Concurrent < lock-free < wait-free
  • 66. class CASBackedQueue[A] { val _queue = new AtomicReference(Vector[A]()) // does not block, may spin though @tailrec final def put(a: A): Unit = { val queue = _queue.get val appended = queue :+ a if (!_queue.compareAndSet(queue, appended)) put(a) } } Concurrent < lock-free < wait-free
  • 67. class CASBackedQueue[A] { val _queue = new AtomicReference(Vector[A]()) // does not block, may spin though @tailrec final def put(a: A): Unit = { val queue = _queue.get val appended = queue :+ a if (!_queue.compareAndSet(queue, appended)) put(a) } } Concurrent < lock-free < wait-free
  • 68. Concurrent < lock-free < wait-free
  • 69. Concurrent < lock-free < wait-free
  • 70. Concurrent < lock-free < wait-free
  • 71. Concurrent < lock-free < wait-free
  • 72. Concurrent < lock-free < wait-free
  • 73. Concurrent < lock-free < wait-free
  • 74. Concurrent < lock-free < wait-free “concurrent” data structure < lock-free* data structure < wait-free data structure * Both versions are used: lock-free / lockless
  • 75. Concurrent < lock-free < wait-free Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms
 Maged M. Michael Michael L. Scott An algorithm is wait-free if every operation has a bound on the number of steps the algorithm will take before the operation completes.
  • 76. wait-free: j.u.c.ConcurrentLinkedQueue Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms
 Maged M. Michael Michael L. Scott public boolean offer(E e) { checkNotNull(e); final Node<E> newNode = new Node<E>(e); for (Node<E> t = tail, p = t;;) { Node<E> q = p.next; if (q == null) { // p is last node if (p.casNext(null, newNode)) { // Successful CAS is the linearization point // for e to become an element of this queue, // and for newNode to become "live". if (p != t) // hop two nodes at a time casTail(t, newNode); // Failure is OK. return true; } // Lost CAS race to another thread; re-read next } else if (p == q) // We have fallen off list. If tail is unchanged, it // will also be off-list, in which case we need to // jump to head, from which all live nodes are always // reachable. Else the new tail is a better bet. p = (t != (t = tail)) ? t : head; else // Check for tail updates after two hops. p = (p != t && t != (t = tail)) ? t : q; } } This is a modification of the Michael & Scott algorithm, adapted for a garbage-collected environment, with support for interior node deletion (to support remove(Object)). 
 For explanation, read the paper.
  • 77. Lesson learned: Different ways to construct concurrent data structures. Pick the right impl for the right job.
  • 78. I / O
  • 80. IO / AIO / NIO
  • 81. IO / AIO / NIO / Zero
  • 82. Synchronous I / O — Havoc Pennington (HAL, GNOME, GConf, D-BUS, now Typesafe) When I learned J2EE about 2008 with some of my desktop colleagues our reactions included something like:
 
 ”wtf is this sync IO crap, 
 where is the main loop?!” :-)
  • 89. Kernels and CPUs […] switching from user-level to kernel-level 
 on a (2.8 GHz) P4 is 1348 cycles. 
 
 […] Counting actual time, the P4 takes 481ns […] http://wiki.osdev.org/Context_Switching
  • 90. I / O
  • 91. I / O
  • 92. I / O
  • 93. I / O
  • 94. I / O
  • 95. I / O “Don’t worry. It only gets worse!”
  • 96. I / O “Don’t worry. It only gets worse!” Same data in 3 buffers!4 mode switches!
  • 97. Asynchronous I / O [Linux] Linux AIO = JVM NIO
  • 98. Asynchronous I / O [Linux] NewIO… since 2004! (No-one calls it “new” any more) Linux AIO = JVM NIO
  • 99. Asynchronous I / O [Linux] Less time wasted waiting. Same amount of buffer copies.
  • 100. ZeroCopy = sendfile [Linux] “Work smarter. Not harder.” http://fourhourworkweek.com/
  • 102. ZeroCopy = sendfile [Linux] Data never leaves kernel mode!
  • 104. Lesson learned: By picking the right tool, you can avoid wasting CPU time. Asynchronous IO will not beat a synchronous seek throughput-wise but it can allow higher scalability.
  • 106. C10K and beyond “10.000 concurrent connections” Not a new problem, pretty old actually: ~12 years old. http://www.kegel.com/c10k.html
  • 107. C10K and beyond- why? It’s not about performance. It’s about scalability. These are orthogonal things. Threading differences: apache / nginx
  • 112. epoll
  • 113. C10K – epoll [Linux]
  • 114. C10K – epoll [Linux]
  • 115. C10K – epoll [Linux]
  • 116. C10K – epoll [Linux]
  • 117. C10K – epoll [Linux]
  • 118. C10K O(n) is a no-go for epic scalability.
  • 119. C10K O(n) is a no-go for epic scalability. State of Linux scheduling: O(n) O(1) CFS (O(1)/ O(log n)) And Socket selection: Select/Poll O(n) EPoll (O(1))
  • 120. C10K O(n) is a no-go for epic scalability. State of Linux scheduling: O(n) O(1) CFS (O(1)) And Socket selection: Select/Poll O(n) EPoll (O(1)) O(1) IS a go for epic scalability. Moral:
  • 122. Distributed Systems “… in which the failure of a computer you didn't even know existed can render your own computer unusable.” — Leslie Lamport http://research.microsoft.com/en-us/um/people/lamport/pubs/distributed-system.txt
  • 123. Distributed Systems The bigger the system, the more “random” latency / failure noise. Embrace instead of hiding it.
  • 125. Backup requests A technique for fighting “long tail latencies”. By issuing duplicated work, when SLA seems in danger.
  • 131. Backup requests Avg Std dev 95%ile 99%ile 99.9%ile No backups 33 ms 1524 ms 24 ms 52 ms 994 ms After 10ms 14 ms 4 ms 20 ms 23 ms 50 ms After 50ms 16 ms 12 ms 57 ms 63 ms 68 ms Jeff Dean - Achieving Rapid Response Times in Large Online Services Peter Bailis - Doing Redundant Work to Speed Up Distributed Queries Akka - Krzysztof Janosz @ Akkathon, Kraków - TailChoppingRouter (docs, pr)
  • 132. Lesson learned: Backup requests allow trade-off increased load for decreased latency.
  • 134. Combined requests & back-pressure
  • 135. Combined requests & back-pressure No no no…! Not THAT Back-pressure!
  • 136. Combined requests & back-pressure THAT kind of back-pressure: www.reactive-streams.org
  • 137. Combined requests & back-pressure THAT kind of back-pressure: www.reactive-streams.org
  • 138. Combined requests & back-pressure THAT kind of back-pressure: www.reactive-streams.org
  • 139. Combined requests & back-pressure THAT kind of back-pressure: www.reactive-streams.org
  • 140. Combined requests A technique for avoiding duplicated work. By aggregating requests, possibly increasing latency. “Wat?Why would I increase latency!?”
  • 145. Lesson learned: Back-pressure saves systems from overload. Combined requests trade higher latency, for less work for the downstream.
  • 147. Wrapping up You don’t need to be a great mechanic to be a great racing driver, but you must understand how your bolid works! ~ Martin Thompson (in context of Mechanical Sympathy)
  • 148. Wrapping up • Someone has to bite the bullet though! • We’re all running on real hardware. • Libraries do it so you don’t have to - pick the right one! Be aware that:
  • 149. Wrapping up • Keep your apps pure • Be aware of internals • Async all the things! • Messaging all the way! • Someone has to bite the bullet though! • We’re all running on real hardware. • Libraries do it so you don’t have to - pick the right one! Be aware that:
  • 150. Links • akka.io • reactive-streams.org • akka-user
 • Gil Tene - How NOT to measure latency, 2013 • Jeff Dean @Velocity 2014 • Alan Bateman, Jeanfrancois Arcand (Sun) Async IO Tips @ JavaOne • http://linux.die.net/man/2/select • http://linux.die.net/man/2/poll • http://linux.die.net/man/4/epoll • giltene/jHiccup • Linux Journal: ZeroCopy I, Dragan Stancevis 2013 • Last slide car picture: http://actu-moteurs.com/sprint/gt-tour/jean- philippe-belloc-un-beau-challenge-avec-le-akka-asp-team/2000
  • 151. Links • http://wiki.osdev.org/Context_Switching • CppCon: Herb Sutter "Lock-Free Programming (or, Juggling Razor Blades)" • http://www.infoq.com/presentations/reactive-services-scale • Gil Tene’s HdrHistogram.org • http://hdrhistogram.github.io/HdrHistogram/plotFiles.html • Rob Pike - Concurrency is NOT Parallelism (video) • Brendan Gregg - Systems Performance: Enterprise and the Cloud (book) • http://psy-lob-saw.blogspot.com/2015/02/hdrhistogram-better-latency-capture.html • Jeff Dean, Luiz Andre Barroso - The Tail at Scale (whitepaper,ACM) • http://highscalability.com/blog/2012/3/12/google-taming-the-long-latency-tail-when- more-machines-equal.html • http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system- architects/ • Marcus Lagergren - Oracle JRockit:The Definitive Guide (book) • http://mechanical-sympathy.blogspot.com/2013/08/lock-based-vs-lock-free- concurrent.html • Handling of Asynchronous Events - http://www.win.tue.nl/~aeb/linux/lk/lk-12.html • http://www.kegel.com/c10k.html
  • 152. Links • www.reactivemanifesto.org/ • Seriously the only right way to micro benchmark on the JVM: • JMH openjdk.java.net/projects/code-tools/jmh/ • JMH for Scala: https://github.com/ktoso/sbt-jmh • http://www.ibm.com/developerworks/library/l-async/ • http://lse.sourceforge.net/io/aio.html • https://code.google.com/p/kernel/wiki/AIOUserGuide • ShmooCon: C10M - Defending the Internet At Scale (Robert Graham) • http://blog.erratasec.com/2013/02/scalability-its-question-that-drives-us.html#.VO6E11PF8SM •User-level threads....... with threads. - Paul Turner @ Linux Plumbers Conf 2013 •https://www.facebook.com/themainstreetpiggyboys/photos/a. 1390784047896753.1073741829.1390594984582326/1423592294615928/? type=1&theater for the Rollin’ Cuy on last slide
  • 153. Special thanks to: • Aleksey Shipilëv • Andrzej Grzesik • Gil Tene • Kirk Pepperdine • Łukasz Dubiel • Marcus Lagergren • Martin Thompson • Mateusz Dymczyk • Nitsan Wakart Thanks guys, you’re awesome. alphabetically, mostly • Peter Lawrey • Richard Warburton • Roland Kuhn • Sergey Kuksenko • Steve Poole • Viktor Klang a.k.a. √ • Antoine de Saint Exupéry;-) • and the entire AkkaTeam • the Mechanical Sympathy Mailing List
  • 154. Learn more at: • SCKRK.com – • Java.pl – Polish (Kraków) JUG • KrakowScala.pl – Kraków Scala User Group • LambdaKRK.pl – Lambda Lounge Kraków • GeeCON.org – awesome “all around the JVM” conference, 
 as always: Kraków May 2015 • GeeCON Tricity (at the beach!) - September 2015 • GeeCON Prague - October 2015 Software Craftsmanship Kraków Computer Science Whitepaper Reading Club Kraków
  • 155. Thanks & Q/A! ktoso @ typesafe.com twitter: ktosopl github: ktoso team blog: letitcrash.com home: akka.io
  • 156. ©Typesafe 2015 – All Rights Reserved