Reactive Slick
The principles behind Slick 3
Lutz Hühnken (@lutzhuehnken)
JDBC
   try {
      // make the connection
      Class.forName(driver)
      connection = DriverManager.getConnection(url,...
Reactive Slick
JDBC
The problem with that code
• well there are probably many, style etc.
But the problem:
• Synchronous I...
Reactive Slick
Digression: Concurrency model
Java EE: Threads. Servlet API: Thread per Request
4
Note: This is a snapshot ...
Reactive Slick
Problems with Threads
• Context Switching is expensive
• Memory overhead per thread
•  Lock contention when...
Reactive Slick
Digression: Concurrency model
6
Source: John Rose, Java VM Architect, JFokus, Stockholm, February 2015
Reactive Slick
Task level concurrency
This does not mean: „no threads“.
It means: Threads should not be the finest level o...
Reactive Slick
Digression: Concurrency model
Reactive: Sub-Thread. Play: n threads per m requests
8
Note: This is a snapsh...
Reactive Slick
Solution: Use Future / blocking combo
• Put blocking database calls in
Future(blocking( ... ))
• Contention...
Reactive Slick
Better solution: Isolation
10
Note: This is a snapshot at one point in time, not a sequence of events.
Reactive Slick
New Question: Size
Idea: number of threads <= number of connections in pool, so
getConnection never blocks....
Reactive Slick
It was a trick question…
A formula which has held up pretty well across a
lot of benchmarks for years is th...
Reactive Slick
Example: Quad-core server
13
Image by Stefan Zeiger
Reactive Slick
The difference
14
Image by Stefan Zeiger
Reactive Slick
Slick
Every Database contains an AsyncExecutor that manages the
thread pool for asynchronous execution of D...
Switch to code
Reactive Slick
JDBC (again, it’s the same slide)
   try {
      // make the connection
      Class.forName(driver)
      c...
Reactive Streams
Reactive Slick
Reactive Streams Overview
• How do we:
• Handle potentially infinite streams of data?
• Handle data in a re...
Reactive Slick
Supply and Demand
• Data Items Flow Downstream
• Demand Flows Upstream
• Data Items flow only when there is...
Reactive Slick
Reactive Streams Specification
• Interface Specification
• Java interfaces for implementations
• TCK
• Test...
Reactive Slick
Publisher
package org.reactivestreams;
public interface Publisher<T> {
public void subscribe<T>(
Subscriber...
Reactive Slick
Subscriber
public interface Subscriber<T> {
public void onSubscribe(
Subscription subscription);
public voi...
Reactive Slick
Subscription
public interface Subscription {
public void cancel();
public void request(long elements);
}
24
Reactive Slick
Dynamic Push-Pull
• “Push” behavior when consumer is faster
• “Pull” behavior when producer is faster
• Swi...
Reactive Slick
Linear Transformations
26
Demand
Source Sinkmap …
Reactive Slick
Linear Transformations
27
Source
Deman
Sink
drop
map
Reactive Slick
Linear Stream Transformations
• Deterministic (like for collections)
• map, filter, collect, grouped, drop,...
Reactive Slick
Non-linear Stream Transformations
• Fan-In
• merge, concat, zip
• Fan-Out
• broadcast, route, unzip
29
Reactive Slick
Fan In
30
Reactive Slick
Fan Out
31
Reactive Slick
Materialization
• Akka Streams separate the what from the how
• declarative Source/Flow/Sink DSL to create ...
Switch to code
Reactive Slick
Try it yourself!
34
Reactive Slick
Thank You
Further information:
http://slick.typesafe.com
Contact:
lutz.huehnken@typesafe.com
Twitter:
@lutz...
©Typesafe 2015 – All Rights Reserved
Nächste SlideShare
Wird geladen in …5
×

Reactive Slick at Scala User Group Vienna May 20 2015

696 Aufrufe

Veröffentlicht am

The Principles behind Slick 3

The code shown in the talk (not on the slides) was mostly from https://www.typesafe.com/activator/template/hello-slick-3.0

Veröffentlicht in: Software
0 Kommentare
0 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Keine Downloads
Aufrufe
Aufrufe insgesamt
696
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
5
Aktionen
Geteilt
0
Downloads
12
Kommentare
0
Gefällt mir
0
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Reactive Slick at Scala User Group Vienna May 20 2015

  1. 1. Reactive Slick The principles behind Slick 3 Lutz Hühnken (@lutzhuehnken)
  2. 2. JDBC    try {       // make the connection       Class.forName(driver)       connection = DriverManager.getConnection(url, username, password)         // create the statement, and run the select query       val statement = connection.createStatement()       val resultSet = statement.executeQuery("SELECT host, user FROM user")       while ( resultSet.next() ) {         val host = resultSet.getString("host")         val user = resultSet.getString("user")         println("host, user = " + host + ", " + user)       }     } catch {       case e => e.printStackTrace     }     connection.close()   } 2
  3. 3. Reactive Slick JDBC The problem with that code • well there are probably many, style etc. But the problem: • Synchronous I/O • Will block the thread it’s running on. • This is true not only for JDBC, but any DB lib running on top of JDBC, Slick included. We like to avoid that. Why? 3
  4. 4. Reactive Slick Digression: Concurrency model Java EE: Threads. Servlet API: Thread per Request 4 Note: This is a snapshot at one point in time, not a sequence of events.
  5. 5. Reactive Slick Problems with Threads • Context Switching is expensive • Memory overhead per thread •  Lock contention when communicating between threads (if you have shared mutable state, which is likely..) 5
  6. 6. Reactive Slick Digression: Concurrency model 6 Source: John Rose, Java VM Architect, JFokus, Stockholm, February 2015
  7. 7. Reactive Slick Task level concurrency This does not mean: „no threads“. It means: Threads should not be the finest level of concurrency. Not be the level the application developer works on. 7
  8. 8. Reactive Slick Digression: Concurrency model Reactive: Sub-Thread. Play: n threads per m requests 8 Note: This is a snapshot at one point in time, not a sequence of events.
  9. 9. Reactive Slick Solution: Use Future / blocking combo • Put blocking database calls in Future(blocking( ... )) • Contention for Connections (but may be limited by the ExecutionContext) • A saturated thread pool blocks everything. 9
  10. 10. Reactive Slick Better solution: Isolation 10 Note: This is a snapshot at one point in time, not a sequence of events.
  11. 11. Reactive Slick New Question: Size Idea: number of threads <= number of connections in pool, so getConnection never blocks. Given you want to handle 10.000 concurrent clients.. what size should your connection pool be? • 10 • 100 • 1.000 • 10.000 11
  12. 12. Reactive Slick It was a trick question… A formula which has held up pretty well across a lot of benchmarks for years is that for optimal throughput the number of active connections should be somewhere near ((core_count * 2) + effective_spindle_count). Core count should not include HT threads, even if hyperthreading is enabled. Effective spindle count is zero if the active data set is fully cached, and approaches the actual number of spindles as the cache hit rate falls. ... There hasn't been any analysis so far regarding how well the formula works with SSDs. 12 (From PostgreSQL)
  13. 13. Reactive Slick Example: Quad-core server 13 Image by Stefan Zeiger
  14. 14. Reactive Slick The difference 14 Image by Stefan Zeiger
  15. 15. Reactive Slick Slick Every Database contains an AsyncExecutor that manages the thread pool for asynchronous execution of Database I/O Actions. 15 mydb = { dataSourceClass = "org.postgresql.ds.PGSimpleDataSource" properties = { databaseName = "mydb" user = "myuser" password = "secret" } numThreads = 10 }
  16. 16. Switch to code
  17. 17. Reactive Slick JDBC (again, it’s the same slide)    try {       // make the connection       Class.forName(driver)       connection = DriverManager.getConnection(url, username, password)         // create the statement, and run the select query       val statement = connection.createStatement()       val resultSet = statement.executeQuery("SELECT host, user FROM user")       while ( resultSet.next() ) {         val host = resultSet.getString("host")         val user = resultSet.getString("user")         println("host, user = " + host + ", " + user)       }     } catch {       case e => e.printStackTrace     }     connection.close()   } 17
  18. 18. Reactive Streams
  19. 19. Reactive Slick Reactive Streams Overview • How do we: • Handle potentially infinite streams of data? • Handle data in a reactive manner? • Achieve asynchronous non-blocking data flow? • Avoid out of memory errors? 19
  20. 20. Reactive Slick Supply and Demand • Data Items Flow Downstream • Demand Flows Upstream • Data Items flow only when there is demand. 20
  21. 21. Reactive Slick Reactive Streams Specification • Interface Specification • Java interfaces for implementations • TCK • Test Harness to validate implementations • Specification Website • http://www.reactive-streams.org/ 21
  22. 22. Reactive Slick Publisher package org.reactivestreams; public interface Publisher<T> { public void subscribe<T>( Subscriber<T> subscriber); } 22
  23. 23. Reactive Slick Subscriber public interface Subscriber<T> { public void onSubscribe( Subscription subscription); public void onNext(T element); public void onComplete(); public void onError(Throwable cause); } 23
  24. 24. Reactive Slick Subscription public interface Subscription { public void cancel(); public void request(long elements); } 24
  25. 25. Reactive Slick Dynamic Push-Pull • “Push” behavior when consumer is faster • “Pull” behavior when producer is faster • Switches automatically between these • Batching demand allows batching data 25
  26. 26. Reactive Slick Linear Transformations 26 Demand Source Sinkmap …
  27. 27. Reactive Slick Linear Transformations 27 Source Deman Sink drop map
  28. 28. Reactive Slick Linear Stream Transformations • Deterministic (like for collections) • map, filter, collect, grouped, drop, take, groupBy, ... • Time-Based • takeWithin, dropWithin, groupedWithin, ... • Rate-Detached • expand, conflate, buffer, ... • asynchronous • mapAsync, mapAsyncUnordered, ... 28
  29. 29. Reactive Slick Non-linear Stream Transformations • Fan-In • merge, concat, zip • Fan-Out • broadcast, route, unzip 29
  30. 30. Reactive Slick Fan In 30
  31. 31. Reactive Slick Fan Out 31
  32. 32. Reactive Slick Materialization • Akka Streams separate the what from the how • declarative Source/Flow/Sink DSL to create blueprint • FlowMaterializer turns this into running Actors 32
  33. 33. Switch to code
  34. 34. Reactive Slick Try it yourself! 34
  35. 35. Reactive Slick Thank You Further information: http://slick.typesafe.com Contact: lutz.huehnken@typesafe.com Twitter: @lutzhuehnken 35
  36. 36. ©Typesafe 2015 – All Rights Reserved

×