Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
•
3 gefällt mir•1,104 views
Intense 3 hour workshop covering Akka Actors, Cluster, Streams, HTTP and more. Including very advanced patterns. Presented with Henrik Engstrom at O'Reilly Software Architecture Conference in New York City in 2017
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
Ähnlich wie Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC (20)
Mehr von Konrad Malawski
Mehr von Konrad Malawski (12)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
- 1. Building a Reactive System with Akka Konrad Malawski (@ktosopl) - Akka Team Henrik Engström (@h3nk3) - Telemetry Team O’Reilly Software Architecture Conference, NYC - April 2016
- 2. Free e-book and printed report. bit.ly/why-reactive Covers what reactive actually is. Implementing in existing architectures. Thoughts from the team that’s building reactive apps since more than 6 years.
- 4. • “TRADITIONAL” AND REACTIVE APPLICATIONS • ACTOR MODEL AND DISTRIBUTED PROGRAMMING • INTRODUCTION TO "BRACES" • AKKA CLUSTERING AND PERSISTENCE • AKKA STREAMS • AKKA HTTP • TESTING AKKA Actors / Streams / HTTP • WHAT FEATURES ARE COMING SOON? AGENDA
- 6. IT IS 2017 AND WE STILL USE • Synchronous local and remote calls • Single machine apps - scaling is an afterthought • Non resilient approaches Result: brittle, slow, non-scalable applications
- 7. WHAT CAN WE DO ABOUT IT? Use message driven/asynchronous programming
- 9. REACTIVE MANIFESTO http://www.reactivemanifesto.org/ • Created in September 2014, +18k signatures • Consists of four traits • Responsive • Resilient • Elastic • Message Driven
- 10. RESPONSIVE
- 11. RESILIENT
- 12. E L A S T I C
- 13. MESSAGE DRIVEN
- 14. The many meanings of Reactive reactivemanifesto.org
- 17. How to think about these techniques? bit.ly/why-reactive
- 18. Akka
- 19. Akka is the Enabler of Reactive Systems • Individual entities, actors, that can contain state • Communication done by message passing • Lock-free concurrency • Loosely coupled and distributable • Fault tolerance via Supervision
- 20. Akka is a Toolkit • A Toolkit, not a Framework • Multitude modules and components – “pick-and-choose” • Performance and distribution always a goal • Resilient and asynchronous from its very core
- 22. What is an Actor? • Behavior (processing) - An actor reacts on messages it receives • State (storage) - An actor is shielded from the rest of the world - no need for synchronization! • Communication - An actor interacts with other actors exclusively via messages • "One actor is no actor" - they come in systems
- 23. Anatomy of an Akka Actor
- 24. Anatomy of an ActorSystem
- 26. VOCABULARY • Drone - autonomous, field-deployed UAV, sends metrics/position to backend system • DroneShadow - backend “mirror” of field-deployed Drone, keeps metrics and represents drone in backend model • Backend - single MicroService, backed by Akka Cluster for resilience/load-balancing
- 27. VOCABULARY • Micro-service - has a single responsibility, it absolutely does not mean “one node”! • Distributed Journal - backing datastore of the single service, often Cassandra, SQL or similar. Service should “own your data.”
- 31. LET’S CODE!
- 32. • Message passing (events are a kind of message) • Distribution via Location Transparency • Lock-free & simple Concurrency • Very powerful abstraction underneath everything we’ll talk about Akka Actors:
- 35. LET’S CODE!
- 39. FAILURE SCENARIO: State Recovery (Akka Persistence)
- 41. http://developer.lightbend.com/docs/akka-commercial-addons/1.0.1/split-brain-resolver.html // application.conf akka.cluster.downing-provider-class = "com.lightbend.akka.sbr.SplitBrainResolverProvider" • Static Quorum - static, very predictable (like zk) • Keep Majority - dynamic, supports dynamic growth • Keep Oldest - keep “most stable” members • Keep Referee - keep “precious side” Split Brain / Failure Detection
- 42. http://developer.lightbend.com/docs/akka-commercial-addons/1.0.1/split-brain-resolver.html • Simple timeout - known as “auto down” • very naive, • not really intended for real-world usage. Naive Failure Detection
- 43. • Battle-tested Membership Protocol • Powerful distribution mechanism • Application aware Sharding • Scale-out & Resilience Akka Cluster:
- 44. • EventSourcing style persistence • Pluggable Journals (Cassandra, SQL, Mongo, Dynamo, and more …) • Trivial to plug in • Enables CQRS-style architectures • Powers AtLeastOnceDelivery Akka Persistence:
- 45. AKKA STREAMS
- 46. Fast Publisher Slow Subscriber Akka Streams - BackPressure
- 47. Subscriber usually has some kind of buffer. Push model
- 48. Push model
- 49. Push model
- 50. Push model
- 51. What if the buffer overflows? Push model
- 52. Use bounded buffer, drop messages + require re-sending Push model
- 53. Kernel does this! Routers do this! (TCP) Use bounded buffer, drop messages + require re-sending Push model
- 54. Increase buffer size… Well, while you have memory available! Push model
- 55. Reactive Streams explained Reactive Streams explained in 1 slide
- 56. Fast Publisher will send at-most 3 elements. This is pull-based-backpressure. Reactive Streams: “dynamic push/pull”
- 57. JEP-266 – soon…! public final class Flow { private Flow() {} // uninstantiable @FunctionalInterface public static interface Publisher<T> { public void subscribe(Subscriber<? super T> subscriber); } public static interface Subscriber<T> { public void onSubscribe(Subscription subscription); public void onNext(T item); public void onError(Throwable throwable); public void onComplete(); } public static interface Subscription { public void request(long n); public void cancel(); } public static interface Processor<T,R> extends Subscriber<T>, Publisher<R> { } }
- 58. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries
- 59. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries Argh, implementing a correct RS Publisher or Subscriber is so hard!
- 60. 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries Reactive Streams: goals Argh, implementing a correct RS Publisher or Subscriber is so hard!
- 61. Reactive Streams: goals Argh, implementing a correct RS Publisher or Subscriber is so hard! You should be using Akka Streams instead! 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries
- 63. LET’S CODE!
- 64. • Fully Typed API • Asynchronous Back-Pressure • First impl. to pass Reactive Streams standard • Reactive Streams coming to JDK9 • Powerful composable ScalaDSL and JavaDSL • Open Materializer API (e.g. Intel GearPump) Akka Streams:
- 65. AKKA HTTP
- 67. A core feature not obvious to the untrained eye…! Quiz time! TCP is a ______ protocol? Akka HTTP
- 68. A core feature not obvious to the untrained eye…! Quiz time! TCP is a STREAMING protocol! Akka HTTP
- 69. Streaming in Akka HTTP http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse]
- 70. Streaming in Akka HTTP http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _]
- 71. Streaming in Akka HTTP http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] Websocket connection as a: Flow[ws.Message, ws.Message]
- 72. Streaming from Akka HTTP (Java) public static void main(String[] args) { final ActorSystem system = ActorSystem.create(); final Materializer materializer = ActorMaterializer.create(system); final Http http = Http.get(system); final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world")); final Route tweetsRoute = path("tweets", () -> completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json()) ); final Flow<HttpRequest, HttpResponse, NotUsed> handler = tweetsRoute.flow(system, materializer); http.bindAndHandle(handler, ConnectHttp.toHost("localhost", 8080), materializer ); System.out.println("Running at http://localhost:8080"); }
- 73. LET’S CODE!
- 74. It’s turtles buffers all the way down!
- 75. Streaming from Akka HTTP
- 76. Streaming from Akka HTTP
- 77. Streaming from Akka HTTP No demand from TCP = No demand upstream = Source won’t generate tweets => Bounded memory stream processing!
- 86. • “Streaming-first” HTTP Server • Powerful composable ScalaDSL and JavaDSL • Built completely on Akka Streams • Trivially exposes TCP level flow control to Akka Streams as backpressure • Simple inter-op with Actors, Futures, Streams • HTTP/2 coming very soon Akka HTTP:
- 87. TESTING ASYNCHRONOUS CODE WITH AKKA
- 88. TestKits and tools provided • Asynchronous is hard • Testing asynchonous code is hard • We’ve prepared plenty tools to make it simple
- 89. TestKits and tools provided • Akka Actors - TestKit • Akka Streams - TestSource / TestSink
- 90. LET’S CODE!
- 91. • All major modules come with dedicated TestKit • Actors, Streams, MultiJVM testing (!) • Makes asynchronous code simpler to test Akka Testing:
- 93. Alpakka is a community driven project • Community of Akka Streams “Connectors” • Akka Streams == Reactive Streams impl == everyone profits! • Similar in goals to Apache Camel • “inter-op all the things!” • We’ve prepared plenty tools to make it simple
- 94. Alpakka – a community for Stream connectors http://developer.lightbend.com/docs/alpakka/current/
- 95. Alpakka – a community for Stream connectors http://developer.lightbend.com/docs/alpakka/current/ Google Pub/Sub … … … … … ……
- 96. Alpakka – a community for Stream connectors http://developer.lightbend.com/docs/alpakka/current/
- 97. Alpakka – a community for Stream connectors http://developer.lightbend.com/docs/alpakka/current/
- 98. LET’S CODE!
- 99. • Vibrant community of Akka Streams developers • Tens of integrations already, more coming as-we-speak • All fully Streaming and Back-pressure Aware and s Akka “Alpakka”:
- 100. MONITORING
- 101. MONITORING FEATURES (2017-04) • Akka Actors • Akka Remoting and Clustering • Lagom Circuit Breakers • Dispatchers / Thread Pools • Various backend integration (ES, StatsD, …) • Sandbox environment (EKG) for easy exploration
- 102. Datadog Integration
- 103. • Commercial monitoring • Get the full picture about your applications in production • Highly optimised, fine-tuned by core Lightbend teams Lightbend Monitoring:
- 104. SUMMING UP
- 105. ARCHITECTURE • Resilient & Elastic from the ground up • High-performance & Asynchronous all-the-way • Highly Self-Healing / Resilient • Well established API endpoints (HTTP/WS/SSE) • Highly reusable & composable code • See Reactive Streams (j.u.c.Flow in JDK9)
- 106. CODE • Full feature parity between Java & Scala DSL (always!) • Asynchronous all-the-way • Understandable and well-defined failure scenarios • Start local, go distributed with no or minimal changes • Highly reusable & composable code • Brought to you by leaders of Reactive Streams
- 107. MONITORING • Awesome monitoring of asynchronous code • Coming soon: • Tracing across Actors, Futures, HTTP • Smart Cluster Sharding insights
- 108. WHAT FEATURES ARE COMING SOON?
- 109. Next steps for Akka New Akka Remoting (benchmarked 1,000,000+ msg/s (!)), (built using Akka Streams, Aeron) More integrations for Akka Streams stages, project Alpakka. Akka Typed actively progressing (!). Akka HTTP/2 Proof of Concept in progress. Akka HTTP as default backend of Play Framework. Highly Available CRDTs with Akka Distributed Data.
- 110. We <3 contributions • Easy to contribute: • https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to- contribute • https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to- have+%28low-prio%29%22 • Akka: akka.io && github.com/akka • Reactive Streams: reactive-streams.org • Mailing list: • https://groups.google.com/group/akka-user • Public chat rooms: • http://gitter.im/akka/dev developing Akka • http://gitter.im/akka/akka using Akka
- 111. Lightbend booth / sponsor area O’Reilly’s “Ask the Experts” Henrik’s talk on Wednesday @ 4.50 pm… Catch us here:Catch us here
- 112. EOF Q/A Konrad Malawski (@ktosopl) - Akka Team Henrik Engström (@h3nk3) - Telemetry Team