20181025 integration reloaded_integration_solutions_based_on_reactive_principles
•
0 gefällt mir•107 views
Status Quo Integration - Reactive Patterns - Architectural thoughts - Use case
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (18)
Ähnlich wie 20181025 integration reloaded_integration_solutions_based_on_reactive_principles
Ähnlich wie 20181025 integration reloaded_integration_solutions_based_on_reactive_principles (20)
Mehr von OPITZ CONSULTING Deutschland
Mehr von OPITZ CONSULTING Deutschland (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
20181025 integration reloaded_integration_solutions_based_on_reactive_principles
- 1. Red Expert Alliance https://www.redexpertalliance.com Integration Reloaded Integration Solutions Based on Reactive Principles
- 2. Red Expert Alliance https://www.redexpertalliance.com About: Arturo Viveros Arturo Viveros • Principal Middleware Architect at Sysco MW • Oracle ACE • Oracle Dev Champion • Twitter: @gugalnikov • Blogs: ‣http://blog.sysco.no/ ‣https://soamythbusters.wordpress.com ‣http://oracletechnocore.blogspot.no
- 3. Red Expert Alliance https://www.redexpertalliance.com About: Sven Bernhardt Sven Bernhardt • Senior Solution Architect at OPITZ CONSULTING Deutschland GmbH • Oracle ACE • Twitter: @sbernhardt • Blogs: ‣http://thecattlecrew.net ‣https://svenbernhardt.wordpress.com
- 5. Red Expert Alliance https://www.redexpertalliance.com Agenda Status Quo Integration01 02 Use case04 Architectural thoughts03 Summary05 Reactive Patterns
- 6. Red Expert Alliance https://www.redexpertalliance.com What is integration about? • Simplified: Get Data from System A and send it to System B System A System B XX• Message/Protocol Transformation
- 7. Red Expert Alliance https://www.redexpertalliance.com What is integration about? • Message Splitting • Routing System A System C System B
- 8. Red Expert Alliance https://www.redexpertalliance.com ESB solutions as basic architecture element global internal specific External Interfaces – Proxy Servics Integration Logic Internal Interfaces – Business Interfaces Database
- 9. Red Expert Alliance https://www.redexpertalliance.com Limited and expensive Scalability Loadbalancer ESB 1 ESB 2 ESB 3 Database
- 10. Red Expert Alliance https://www.redexpertalliance.com Caught in a technology Trap SOA Cloud Service Fixed Technologies Set <XML /> { „dialect“:“JSON“ }
- 11. Red Expert Alliance https://www.redexpertalliance.com Limited agility • Hard to implement Continuous Integration / Continuous Delivery • Complex infrastructure • Missing automated testing capabilities • Expert knowledge needed • Implementation • Deployment • Error analysis http://geek-and-poke.com/
- 12. Red Expert Alliance https://www.redexpertalliance.com But the world is getting more complex!
- 13. Red Expert Alliance https://www.redexpertalliance.com So we’re Constantly Dealing with Overstretched Integration Components / Patterns • ESB • JMS • Application Server • JCA Adapter • SOAP / XML • Etc. http://geek-and-poke.com/
- 14. Red Expert Alliance https://www.redexpertalliance.com Question How to deal with this new complexity efficiently? (By the way it doesn‘t mean that everything we did before was wrong!)
- 15. Red Expert Alliance https://www.redexpertalliance.com Agenda Status Quo Integration01 02 Use case04 Architectural thoughts03 Summary05 Reactive Patterns
- 16. Red Expert Alliance https://www.redexpertalliance.com Reactive Principles Elastic Responsive Message Driven Resilient Value Means Form https://www.reactivemanifesto.org/
- 17. Red Expert Alliance https://www.redexpertalliance.com Actor model • Advantages: • Useful for realizing reactive patterns • Fault tolerant • Distributed execution • Scalability Source SinkActor autonomous Async - Boundary
- 18. Red Expert Alliance https://www.redexpertalliance.com Adopting Reactive Patterns (i.e. Ivy Pattern)
- 19. Red Expert Alliance https://www.redexpertalliance.com Actor hierarchies Source SinkActor autonomous
- 20. Red Expert Alliance https://www.redexpertalliance.com Actor Balancing Topic Actor Actor Router Queue Queue Actor Actor MasterQueue Actor Actor
- 21. Red Expert Alliance https://www.redexpertalliance.com Reactive Patterns: Backpressure • Systems should be designed and implemented to react assertively under sustained load • This is often neglected although such conditions are usually expected and highly predictable • Negotiating back pressure usually requires some sort of Flow Control • Which in simpler terms is the ability to say “hey, slow down!!”, or even “stop it!!”
- 22. Red Expert Alliance https://www.redexpertalliance.com Non-Blocking Backpressure Concept Source SinkActor autonomous Give me 3 messages! Give me 20 messages!
- 23. Red Expert Alliance https://www.redexpertalliance.com Serveral ways to handle Backpressure • Dynamic push / pull • Static / Dynamic Quotas • Buffers • Ack / Nack • Throttling • Caching • Circuit breaker • Scaling
- 24. Red Expert Alliance https://www.redexpertalliance.com Reactive Patterns: Circuit Breaker https://blog.risingstack.com/designing-microservices-architecture-for-failure/ • Used in the industry to protect electronic components • In software development • Protect resources • Help resources to auto-recover from failures • Prevent hanging operations
- 25. Red Expert Alliance https://www.redexpertalliance.com Reactive Patterns: Bulkhead • Used in the industry to partition a ship into sections • In software development • Segragate resources • Protect limited resources from being exhausted
- 26. Red Expert Alliance https://www.redexpertalliance.com Avoiding single points of failure is key for “reactiveness”
- 27. Red Expert Alliance https://www.redexpertalliance.com Important Concept: a chain is no stronger than its weakest link
- 28. Red Expert Alliance https://www.redexpertalliance.com Agenda Status Quo Integration01 02 Use case04 Architectural thoughts03 Summary05 Reactive Patterns
- 29. Red Expert Alliance https://www.redexpertalliance.com Implementing Reactive Patterns with Akka • Akka Streams • Ideal for Fast Data • Seamless Integration • Abstraction to configure end-to-end Backpressure • Fully Asynchronous & non-blocking • Bulkheads & Circuit Breaker patterns are also supported
- 30. Red Expert Alliance https://www.redexpertalliance.com Implementing Reactive Pattern with Kafka • Distributed streaming platform • Dynamic durable buffer • Horizontally scalable • Load distribution • No data loss settings (Producer, Consumer, Broker) • Quotas
- 31. Red Expert Alliance https://www.redexpertalliance.com Next-gen Integration Platform Sysco reactive Integration platform
- 32. Red Expert Alliance https://www.redexpertalliance.com Demo: Backpressure • Fast Producer, Fast Consumers • Fast Producer, Fast and Slow Consumers • Fast Producer, Fast and Slow Consumers with buffering: OverflowStrategy.dropNew • Fast Producer, Fast and Slow Consumers with buffering: OverflowStrategy.backpressure
- 33. Red Expert Alliance https://www.redexpertalliance.com Implement Integration with Serverless/FaaS • Function-as-a-Service (FaaS) • Atomar step within a certain business capability • Example: Bicycle rental - Process incoming Damage events • Serverless • No need to care about servers • Scale-out functions automatically on-demand • You only pay processing time
- 34. Red Expert Alliance https://www.redexpertalliance.com Implement Integration with Serverless/FaaS System A System B X f(x) getData f(x) sendDataf(x) transformData
- 35. Red Expert Alliance https://www.redexpertalliance.com When is FaaS adequate for integration? • Typical use cases: • Batch processing, which takes place on regular schedules • Event processing • Considerations to make: • Number of invocations • Keep runtime short • Serverless Platforms will terminate long-running Functions • Don‘t block HW resources for too long
- 36. Red Expert Alliance https://www.redexpertalliance.com Integration with Fn Project • Implement functions • F(x) getData • F(x) transformData • F(x) sendData • Orchestrate functions using Fn Flow • Alternatively: Use Topics in between the functions
- 37. Red Expert Alliance https://www.redexpertalliance.com Agenda Status Quo Integration01 02 Use case04 Architectural thoughts03 Summary05 Reactive Patterns
- 38. Red Expert Alliance https://www.redexpertalliance.com New digital business models in bicycle rental [Rainer Klute]
- 39. Red Expert Alliance https://www.redexpertalliance.com Happy Path: Renting a bike Scan a QR Codes at bike rental station with an mobile app Book the rental bike Remove the bicycle from the station and drive off Open lock at rental station
- 40. Red Expert Alliance https://www.redexpertalliance.com Use Case: Bonus Action „After renting a bike for a fee for the 10th time you get a 20% discount and can get a free coffee at the next Mermaid Coffee during your tour at our expense.“
- 41. Red Expert Alliance https://www.redexpertalliance.com What does this mean for the application? • Connection to CRM and ERP necessary • Calculation of different cost rates on the basis of historical data • Connection of all branches of the external partner (Mermaid Coffee) • Near real-time - cyclists must still have rented a bike to redeem their coffee • Payment flows between partners, so that a high degree of robustness is necessary
- 42. Red Expert Alliance https://www.redexpertalliance.com Solution approach: Actors & Reactive Principles Enable bonuses? Buffer Topic Get data from ERP Get data from CRM Inform Partner Inform Custome r Topic
- 43. Red Expert Alliance https://www.redexpertalliance.com Use Case: Damage 3 out of 5 bikes at the same rental station fell over in 2 minutes???
- 44. Red Expert Alliance https://www.redexpertalliance.com Vandalism is the Hulk of our business model [Marcus Pink]
- 45. Red Expert Alliance https://www.redexpertalliance.com Solution approach: Fast Data & Integration Analyze Events Buffer Topic Inform technician Damage events New Vandalism event Process Damage claims Topic
- 46. Red Expert Alliance https://www.redexpertalliance.com Agenda Status Quo Integration01 02 Use case04 Architectural thoughts03 Summary05 Reactive Patterns
- 47. Red Expert Alliance https://www.redexpertalliance.com Summary • Integration is still a relevant discipline nowadys, but the way how to implement it has chnged • Existing integration approaches based on ESB solutions are still valid, but • Businesses getting disrupted, are volatile and techologies is emerging rapidly, so the architecture needs to consider this • Architectures must address this accordingly