Dual write strategies for microservices
•
11 gefällt mir•4,283 views
Patterns covered: Modular monolith Two-phase commit Orchestration Choreography Saga Event sourcing Outbox pattern Parallel pipelines Listen to Yourself Slides from Red Hat Summit 2021 talk https://events.summit.redhat.com/widget/redhat/sum21/sessioncatalog/session/1607126048915001oL4X
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Dual write strategies for microservices
Ähnlich wie Dual write strategies for microservices (20)
Mehr von Bilgin Ibryam
Mehr von Bilgin Ibryam (11)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Dual write strategies for microservices
- 1. Dual Write Strategies for Microservices Bilgin Ibryam @bibryam Product Manager Red Hat
- 2. About me 2 ▸ Product Manager at Red Hat ▸ Former Principal Architect ▸ Committer at Apache Camel ▸ Author ･ Camel Design Patterns ･ Kubernetes Patterns ▸ @ bibryam ▸ https://www.ofbizian.com
- 3. There is a catch 3 Challenges with legacy applications ▸ Frequent deployment is difficult ▸ Obstacles to scaling development ▸ Scaling the application can be difficult Modern application benefits ▸ Greater team autonomy ▸ Reliability, scalability, and other abilities ▸ Reduce time to market
- 4. The dual-write problem 4 Dual-write challenge behind the scene ▸ Updating more than one resources ▸ Reliable inter-service communication ▸ Coordinating long-running, business transactions ▸ Recovery from failed distributed transaction ▸ Implementing idempotent operations ▸ Scaling with increasing data volume
- 7. Modular monolith 7 Challenges ▸ Strong data consistency requirements ▸ Strict performance requirements ▸ Complex business transactions Solution ▸ Single process deployment unit ▸ Independent modules with clear responsibilities and well-defined interfaces/contracts ▸ Each module access only its tables
- 8. Implementation options 8 Diagram credits @axelfontaine
- 9. Modular monolith 9 Benefits ▸ Simple transaction semantics with local transactions ▸ ACID properties Drawbacks ▸ Shared runtime hinders independent deployment, scalability, and failure isolation ▸ Shared database can lead to coupling among modules
- 10. Two-phase commit
- 11. Two-phase commit 11 Challenges ▸ Strong data consistency requirements ▸ Heterogeneous data sources ▸ Integrating with a third-party or legacy system ▸ Exactly-once message processing Solution ▸ 2PC protocol and X/Open XA specification ▸ JTA and WS AtomicTransaction implementations
- 12. Implementation options 12 Distributed transaction managers ▸ Narayana, JOTM, BTM, Atomikos, MSDTC Datasources ▸ PostgreSQL, MySQL, Db2, Oracle, SQL Server ▸ ActiveMQ, HornetQ, MSMQ, IBM MQ, Solace ▸ Infinispan, Hazelcast Non 2PC/XA distributed transactions ▸ eBay’s GRIT protocol
- 13. Two-phase commit 13 Benefits ▸ An out-of-the-box, standard-based solution ▸ Implemented by many traditional datasources ▸ ACID properties Drawbacks ▸ A blocking protocol with database locks and performance penalty ▸ All participating services must be available ▸ Manual recovery from controller crashes
- 16. Challenges ▸ Business process spanning several services ▸ Business functions lasting for hours or days ▸ A single location for management and monitoring of distributed transactions Solution ▸ Implement each business transaction that spans multiple services as a sequence of local transactions Saga 16
- 17. Saga implementation options 17 Coordination approaches ▸ Orchestration - a controller tells the participants what local transactions to execute ▸ Choreography - business transaction coordination logic is spread among all participants Communication mechanisms ▸ Synchronous, for example HTTP, gRPC ▸ Asynchronous, for example Apache ActiveMQ, Apache Kafka Orchestration Choreography
- 18. Orchestration based Saga 18 Benefits ▸ Doesn’t require all participants to be available at the same time, nor to have knowledge about each other ▸ Single place to define and monitor the transaction flow Drawbacks ▸ Complex programing model, requires coordination, compensation logic, and idempotency implementations ▸ Lacks Isolation from AC*D properties which can cause dirty reads, lost updates, non-repeatable reads
- 19. Choreography
- 20. Choreography 20 Challenges ▸ Business process spanning several services ▸ Business functions lasting for hours or days ▸ Highly scalable and available system Solution ▸ Implement each business transaction that spans multiple services as a sequence of local transactions with distributing decision making in each service
- 21. Choreography 21 Benefits ▸ Doesn't require a transaction coordinator service ▸ Better performance compared to orchestration approach due to smaller number of interactions Drawbacks ▸ No single place to define and monitor the business transaction flow ▸ Coupling among participant with point to point interactions ▸ Lacks Isolation from AC*D properties
- 22. Implementation options 22 Publish, then local-commit Local-commit, then publish Event sourcing Outbox pattern Two-phase commit
- 23. Outbox Pattern 23 Offers an approach for services to update their data store and notify other services in a reliable and eventually consistent manner. Benefits ▸ Addresses the dual-write problem ▸ Offers “read your own writes" semantics Drawbacks ▸ Requires specialized tools, such as Debezium
- 25. Parallel pipelines 25 Challenges ▸ Avoid dual writes to a local database and a messaging system Solution ▸ Publish a message to other services and yourself in the same messaging system
- 26. Listen to Yourself 26 Benefits ▸ Simple, scalable architecture with parallel processing capabilities Drawbacks ▸ Requires temporal dismantling, not commonly applicable ▸ Hard to reason about the global system state ▸ Lacks “read your own writes" semantics
- 27. Summary
- 29. Dual write strategies for microservices 29 Modular Monolith Two-phase Commit Orchestration Choreography Parallel Pipeline Service runtime Single process Single/Multiple Multiple Multiple Multiple Datasources Single Heterogeneous (requires XA) Heterogeneous Heterogeneous Heterogeneous Point of control Centralized Centralized Centralized Distributed Distributed Steps/Flow execution At once At once (for happy paths), highly coupled Sequential, temporal decoupling Sequential, temporal decoupling Parallel, temporal decoupling Properties ACID, blocking, synchronous ACID, blocking, synchronous ACD, non-blocking, (a)synchronous, ACD, non-blocking, asynchronous, ACD, non-blocking, asynchronous, Examples Local transactions XA, JTA, WS-AT Saga/Outbox, Debezium, Kafka Saga/Outbox, Debezium, Kafka Listen to yourself pattern
- 30. 30 Red Hat OpenShift Streams for Apache Kafka a fully managed Apache Kafka service by Red Hat http://red.ht/TryKafka Try Apache Kafka in seconds