Part I - Advanced concepts of Akka Streams.

1. By:
Harmeet Singh (Taara)
Sr. Software Consultant

2. Agenda
1. Flows
2. Operator Fusion.
3. Graph DSL.
4. Leftover.

3. Akka Stream: Flows
The way to perform transformations to the data as it
flows between Source and Sink.

4. Akka Stream: Flows

5. Akka Stream: Example
object StreamingCopyApp extends App {
val spath = Paths.get("/home/harmeet/akka­stream/knolx.log")
val source: Source[ByteString, Future[IOResult]] =
FileIO.fromPath(spath)
val dpath = Paths.get("/home/harmeet/akka­stream­copy")
val sink: Sink[ByteString, Future[IOResult]] = FileIO.toPath(dpath)
val runnableGraph: RunnableGraph[Future[IOResult]] = source.to(sink)
implicit val system = ActorSystem("akka­stream")
implicit val ec = system.dispatcher
implicit val materializer = ActorMaterializer()
runnableGraph.run().foreach { result =>
println(s"${result.status}, ${result.count} bytes read")
system.terminate()
}
}

6. Akka Stream: Flow Example
val frame: Flow[ByteString, String, NotUsed] =
Framing.delimiter(ByteString("n"), 10240).map(_.decodeString("UTF8"))
val parse: Flow[String, Event, NotUsed] = Flow[String].map(Event.parsing)
val filter: Flow[Event, Event, NotUsed] = Flow[Event].filter(_.log ==
"DEBUG")
val serialize: Flow[Event, ByteString, NotUsed] = Flow[Event].map{ event =>
ByteString(event.toJson.compactPrint)
}
val composedFlow: Flow[ByteString, ByteString, NotUsed] = frame.via(parse)
.via(filter)
.via(serialize)
val runnableGraph: RunnableGraph[Future[IOResult]] =
source.via(composedFlow).toMat(sink) (Keep.right)

7. Akka Stream: Flow Example
val composedFlow: Flow[ByteString, ByteString, NotUsed] =
Framing.delimiter(ByteString("n"), 10240)
.map(_.decodeString("UTF8"))
.map(Event.parsing)
.filter(_.log == "DEBUG")
.map { event =>
ByteString(event.toJson.compactPrint)
}
val runnableGraph: RunnableGraph[Future[IOResult]] =
source.via(composedFlow)
.toMat(sink) (Keep.right)

8. Akka Stream: Operator Fusion
Operator Fusion is used to performance optimization to
mitigate the cost of passing elements between the
different stages.
Two main consequences are:
1. Passing elements fused stages faster.
2. Fused stages no longer run in parallel.

9. Akka Stream: Operator Fusion
Q. How can we execute fused stages in parallel???
Ans:
1. By using async method
2. Turn off auto-fusing (akka.stream.materializer.auto­
fusing=off)
Example:
Source(List(1, 2, 3)).map(_ + 1)
.async
.map(_ * 2).to(Sink.ignore)

10. Akka Stream: Operator Fusion

11. Akka Stream: Graph DSL
What if your data flow cannot be modeled as a simple linear process?
What if process is better modeled as computation graph instead?

12. Akka Stream: Graph DSL
The Graph DSL is kind of diagram ASCII art – in many cases
you could translate a white board diagram of a graph into
the DSL.
A graph problem is one that typically involves the concept of
fanning out and fanning in.
●
Fanning out: Single input to multiple output.
●
Fanning in: Multiple inputs single output.

13. Akka Stream: Graph DSL Example
val in = Source(1 to 5)
val f1 = Flow[Int].map(_ * 2)
val f2 = Flow[Int].map(_ * 1)

14. Akka Stream: Graph DSL Example
val f3 = Flow[Int].map(_ * 2)
val f4 = Flow[Int].map(_ + 1)
val out = Sink.foreach[Int](println)

15. Akka Stream: Graph DSL Example
val bcast = builder.add(Broadcast[Int](2))
val merge = builder.add(Merge[Int](2))

16. Akka Stream: Graph DSL Example
in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out

17. Akka Stream: Graph DSL Example
bcast ~> f3 ~> merge

18. Akka Stream: Graph DSL Example
val graph = RunnableGraph.fromGraph(GraphDSL.create() {
implicit builder: GraphDSL.Builder[NotUsed] =>
import GraphDSL.Implicits._
val in = Source(1 to 5)
val out = Sink.foreach[Int](println)
val f1 = Flow[Int].map(_ * 2)
val f2 = Flow[Int].map(_ * 1)
val f3 = Flow[Int].map(_ * 2)
val f4 = Flow[Int].map(_ + 1)
val bcast = builder.add(Broadcast[Int](2))
val merge = builder.add(Merge[Int](2))
in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out
bcast ~> f3 ~> merge
ClosedShape
})

19. Akka Stream: Graph DSL ClosedShape
Thegraph itself wasfully self-contained and complete.
OR
It doesnot haveany open input port and output port like
aclosed circuit.

20. Akka Stream: Partial Graphs
Whenever weneed to createareusablecomponent for
graph, which wecan useaccording to requirementswith
existing or new graph.
https://doc.akka.io/docs/akka/2.5/scala/stream/stream-graphs.html#constructing-and-combinin

21. Leftovers
➔ API’s
➔ Error Handling.
➔ Streaming HTTP
➔ Handling IO Stream
➔ Unit Testing

22. References:
✔ AkkainAction by Raymond Roestenburg, Rob Bakker, Rob Williams.
✔ MasteringAkkaby Christian Baxter
✔ https://doc.akka.io/api/akka/current/akka/stream/index.html?_ga=2.25006541