Akka Streams & HTTP provides reactive, asynchronous, and non-blocking streams for processing data and HTTP requests and responses. It builds upon Akka IO and the Reactive Streams initiative to allow stream-based topologies to be declared and run for tasks like processing big data, serving clients simultaneously with limited resources, and building distributed applications that integrate with external systems over HTTP. Key features include stream sources, sinks, and transformations along with a routing DSL for building HTTP servers and clients on top of Akka IO and HTTP Core.
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Akka Streams and HTTP
1. Akka Streams & HTTP
Dr. Roland Kuhn
@rolandkuhn — Akka Tech Lead
2. Streams Occur Naturally
• traditionally:
• file input/output
• network connections
• more modern:
• processing big data with finite memory
• real-time data processing (CEP)
• serving numerous clients simultaneously with bounded
resources (IoT, streaming HTTP APIs)
2
3. 3
What is a Stream?
• ephemeral, time-dependent sequence of elements
• possibly unbounded in length
• therefore focusing on transformations
«You cannot step twice into the same stream.
For as you are stepping in, other waters are ever
flowing on to you.» — Heraclitus
7. Distributed vs. Static Typing
• in general a very tough problem
• independent entities evolving concurrently
• promising progress on Project Gålbma
• in many specific cases no type evolution necessary
• letting uniform messages flow in safe conduits
• making streams even more interesting
7
13. Origin and motivation
• all participants face the same basic problem
• all are building tools for their community
• a common solution benefits everybody
• interoperability to make best use of efforts
• propose to include in future JDK
13
14. Goals
• minimal interfaces—essentials only
• rigorous specification of semantics
• TCK for verification of implementation
• complete freedom for many idiomatic APIs
• specification should be efficiently implementable
14
15. Reactive Streams
• asynchronous & non-blocking
• flow of data
• flow of demand
• minimal coordination and contention
• message passing allows for distribution across
• applications, nodes, CPUs, threads, actors
15
16. A Data Market using Supply & Demand
• data elements flow downstream
• demand flows upstream
• data elements flow only when there is demand
• data in flight is bounded by signaled demand
• recipient is in control of maximal incoming data rate
16
Publisher Subscriber
data
demand
17. Reactive Push–Pull
• “push”—when consumer is faster
• “pull”—when producer is faster
• switches automatically between these
• batching demand allows batching data
17
Publisher Subscriber
data
demand
21. • TCP for example has it built-in
Back-Pressure can be Propagated
21
CA B
networkhosts
22. The Meat
22
trait Publisher[T] {
def subscribe(sub: Subscriber[T]): Unit
}
trait Subscription {
def request(n: Long): Unit
def cancel(): Unit
}
trait Subscriber[T] {
def onSubscribe(s: Subscription): Unit
def onNext(e: T): Unit
def onError(t: Throwable): Unit
def onComplete(): Unit
}
Not user-level API
Intended as SPI for interop
31. API Design
• goals:
• no magic
• supreme compositionality
• exhaustive model of bounded stream processing
• consequences:
• immutable and reusable stream blueprints
• explicit materialization step
31
http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/stream-design.html
32. Declaring and Running a Stream
32
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)
33. Declaring and Running a Stream
33
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)
34. Declaring and Running a Stream
34
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)
35. Materialization
• Akka Streams separate the what from the how
• declarative Source/Flow/Sink DSL to create blueprint
• FlowMaterializer turns this into running Actors
• this allows alternative materialization strategies
• optimization
• verification / validation
• cluster deployment
• only Akka Actors for now, but more to come!
35
36. Stream Sources
• org.reactivestreams.Publisher[T]
• org.reactivestreams.Subscriber[T]
• Iterator[T] / Iterable[T]
• Code block (function that produces Option[T])
• scala.concurrent.Future[T]
• TickSource
• ActorPublisher
• singleton / empty / failed
• … plus write your own (fully extensible)
36
37. Stream Sinks
• org.reactivestreams.Publisher[T]
• org.reactivestreams.Subscriber[T]
• ActorSubscriber
• scala.concurrent.Future[T]
• blackhole / foreach / fold / onComplete
• … or create your own
37
41. Why do we add an HTTP module?
• Akka is about building distributed applications
• distribution implies integration
• between internal (sub)systems
➜ akka-clusterbasedonakka-remote
• with external systems
➜ akka-http(linguafrancaoftheinternet)
41
42. Akka HTTP—The Origins: Spray.IO
• Fully embeddable HTTP stack based on Actors
• focused on HTTP integration (toolkit)
• server- and client-side
• seamlessly integrated with Akka
42
43. Spray.IO Features
• immutable, case class-based HTTP model
• fast, lightweight HTTP client and server
• powerful DSL for server-side API definition
• fully async & non-blocking, actor-friendly,
modular, testable
• based entirely on Scala & Akka Actors
43
44. Spray.IO Weaknesses
• handling of chunked requests is clunky, incomplete
• dealing with large message entities can be difficult
• some unintuitive corner-cases in routing DSL
• deep implicit argument chains in some cases hard
to debug
• missing features, foremost websocket support
44
49. Akka IO
• bridges Java NIO with Akka Actors or streams
• supports TCP, UDP and SSL
49
50. Akka HTTP Core
• implements HTTP/1.1 according to the RFCs
• based upon the TCP facilities of Akka IO
• exposed as freely reusable stream transformations
• low-level and extensible implementation of
HTTP model and spec
50
51. A Quick View of the HTTP Model
51
case class HttpRequest(
method: HttpMethod = HttpMethods.GET,
uri: Uri = Uri./,
headers: immutable.Seq[HttpHeader] = Nil,
entity: RequestEntity = HttpEntity.Empty,
protocol: HttpProtocol = HttpProtocols.`HTTP/1.1`
) extends HttpMessage
case class HttpResponse(
status: StatusCode = StatusCodes.OK,
headers: immutable.Seq[HttpHeader] = Nil,
entity: ResponseEntity = HttpEntity.Empty,
protocol: HttpProtocol = HttpProtocols.`HTTP/1.1`
) extends HttpMessage
52. Akka HTTP
• builds upon Akka HTTP Core
• adds (un)marshaling for HttpEntities
• adds (de)compression (gzip / deflate)
• high-level server-side routing DSL
• type-safe and flexible
• highly compositional building blocks (Directives)
• includes common behaviors pre-canned
52
54. A Simple Server Example
54
import Directives._
val binding = Http().bind("localhost", 8080)
binding startHandlingWith {
path("order" / HexIntNumber) { id =>
get {
complete(s"Received GET for order $id")
} ~
put {
complete((actor ? Put(id)).mapTo[String])
}
}
}
56. When can we have it?
• currently pre-release versions:
• reactive-streams 1.0.0-RC1
• Akka Streams & HTTP 1.0-M2
• still missing:
• Akka HTTP client features
• SSL integration
• websockets
• Akka Streams & HTTP 1.0 expected end of Feb’15
56