SlideShare ist ein Scribd-Unternehmen logo
1 von 42
Downloaden Sie, um offline zu lesen
Easy Scalability with Akka
Distribute your domain
Who?
● BoldRadius Solutions
○ boldradius.com
○ Typesafe Partner
○ Scala, Akka and Play specialists
○ Ottawa, Saskatoon, San Francisco, Boston, Chicago, Montreal, New
York, Toronto
● Michael Nash, VP Capabilities
○ @MichaelPNash
○ michael.nash@boldradius.com
What?
What will I know in the next 45 minutes?
● Distributed Domain-Driven Design
○ What is it, and how does it apply to project?
○ What is CQRS and ES, and how to they relate?
● Akka
○ How to do DDDD and CQRS/ES with Akka
○ What is different in a DDDD Akka project, and why?
○ How does this kind of project scaled compared to a
“normal” project, even with Akka?
Scalability
● Scalability is not performance
○ Performance increasing reduces response time
○ Scalability increasing improves capacity in relation to
added resources
● Scalability can be...
○ Vertical (Scaling “up”): Bigger box (or boxes)
○ Horizontal (Scaling “out”): More boxes
● The Actor model for the JVM
○ A share-nothing distributed message-
driven compute model
○ Scala and Java APIs
● Like Erlang/Elixir processes, only better
○ Distributed by design, local is an optimization
CQRS and ES
● Command Query Responsibility Segregation
○ Separate the read and write paths of the problem
● Event-Sourcing
○ State is built from a journal of events
● One doesn’t need the other, but they can be
combined very effectively
Event Sourcing
CQRS
Domain
● Your application’s domain is the part that
relates to the problem
● The rest of your application is infrastructure,
basically
● If you have a Payroll app, then Employees,
Benefits, Pay Stubs are in your domain
DDDD
● Domain-Driven Design (DDD) is not only for
Object-Oriented Systems
● Object-Oriented and Functional are not an
either-or choice, or at odds with each other
● Distributed Domain Driven Design is a
natural fit with the actor model
Traditional Domain Instances
Non-Distributed Domain
● Basic CRUD (Create, Read, Update, Delete)
● Writes and reads to same database
○ Always consistent
● Scaled via multiple identical replicas
○ and load balancing on HTTP
● Bottlenecks on contention
○ reads and writes interfere with each other
When we scale the traditional model…
● We add multiple servers
○ so there’s more than one copy of each domain
instance (potentially)
● Each must read state every time
○ from the shared resource (the database)
● Each must write (fully) to the same shared
resource to avoid conflict
● Scalability is limited
Easy Scalability with Akka - Scala Days 2015
With Akka and DDDD
● Wouldn’t it be great if you could just keep
your domain instances in memory?
● But how to recover from its volatile nature:
an event journal!
Easy Scalability with Akka - Scala Days 2015
But I have too many instances!
● Only active domain instances are in memory
● Instances activate and passivate as needed
Activation and Passivation
Ingredients
● Akka - Scala API
○ with Clustering and Persistence modules
● Cassandra
○ And the DataStax driver
● Typesafe Activator
○ For our template
What’s Different?
● Domain object instances are transient, and
memory-resident when in use
● Reads and writes don’t contend for
resources
● Cluster can be grown until domain instances
per node = 1
What do you get?
● Scalability
○ Add nodes to handle higher load on the fly
● Failover
○ No single point of failure or contention
○ Instances can be created anywhere as needed
● Simplicity
○ Build your app without having to worry about scalability
later
Consider your own domain
● Consider writes independently from reads
○ Are they really the same even now?
● Deployment structure is critical to get all
benefits safely
Example Applications
Online Auctions: Two implementations
● Users place bids for online auctions
● Most code shared
● Identical APIs
● Identical deploy structure, same machines,
same number of nodes
● One CRUD, One CQRS/DDDD
CRUD Actor
● Uses Akka
● Receives request
● Writes to datastore directly
CRUD Actor
CQRS/DDDD Actor
● In-memory state
● Recovers state from journal on startup
● Uses become to change states
● Passivates on timeout
Easy Scalability with Akka - Scala Days 2015
Easy Scalability with Akka - Scala Days 2015
Easy Scalability with Akka - Scala Days 2015
Test Setup
3 Cassandra Nodes (Clustered)
3 Akka processing Nodes (Clustered)
1 Front-end (Part of the Akka cluster)
All Small AWS Instances
100 simultaneous users
100 auctions
Two minutes of bidding
Simulation
Response Time Distribution
CRUD CQRS
Scale is different!
Response Time Percentiles: CRUD
Response Time Percentiles: CQRS
Note that the scale is different!
Requests per Second: CRUD
Requests per Second: CQRS
Scale is different!
Total Requests/Responses in 2 mins
CRUD: 75,991
CQRS: 199,357
Adding more nodes...
● CRUD version
○ Contention increases
○ Diminishing benefit to adding more nodes
○ Tuning doesn’t help much
● CQRS version
○ No contention
○ Near-linear benefit until number of nodes = number
of auctions
○ Tuning is very important (type of nodes too…)
Easy Scalability with Akka - Scala Days 2015
Activator Template
https://github.com/boldradius/akka-dddd-template
Add your own domain!
Q & A
michael.nash@boldradius.com
Further Reading

Weitere ähnliche Inhalte

Mehr von BoldRadius Solutions

Introduction to the Typesafe Reactive Platform
Introduction to the Typesafe Reactive PlatformIntroduction to the Typesafe Reactive Platform
Introduction to the Typesafe Reactive PlatformBoldRadius Solutions
 
Towards Reliable Lookups - Scala By The Bay
Towards Reliable Lookups - Scala By The BayTowards Reliable Lookups - Scala By The Bay
Towards Reliable Lookups - Scala By The BayBoldRadius Solutions
 
How You Convince Your Manager To Adopt Scala.js in Production
How You Convince Your Manager To Adopt Scala.js in ProductionHow You Convince Your Manager To Adopt Scala.js in Production
How You Convince Your Manager To Adopt Scala.js in ProductionBoldRadius Solutions
 
Why Not Make the Transition from Java to Scala?
Why Not Make the Transition from Java to Scala?Why Not Make the Transition from Java to Scala?
Why Not Make the Transition from Java to Scala?BoldRadius Solutions
 
String Interpolation in Scala | BoldRadius
String Interpolation in Scala | BoldRadiusString Interpolation in Scala | BoldRadius
String Interpolation in Scala | BoldRadiusBoldRadius Solutions
 
Value Classes in Scala | BoldRadius
Value Classes in Scala | BoldRadiusValue Classes in Scala | BoldRadius
Value Classes in Scala | BoldRadiusBoldRadius Solutions
 
Scala Days Highlights | BoldRadius
Scala Days Highlights | BoldRadiusScala Days Highlights | BoldRadius
Scala Days Highlights | BoldRadiusBoldRadius Solutions
 
What Are For Expressions in Scala?
What Are For Expressions in Scala?What Are For Expressions in Scala?
What Are For Expressions in Scala?BoldRadius Solutions
 
Domain Driven Design Through Onion Architecture
Domain Driven Design Through Onion ArchitectureDomain Driven Design Through Onion Architecture
Domain Driven Design Through Onion ArchitectureBoldRadius Solutions
 
Scala Days 2014: Pitching Typesafe
Scala Days 2014: Pitching TypesafeScala Days 2014: Pitching Typesafe
Scala Days 2014: Pitching TypesafeBoldRadius Solutions
 
Demonstrating Case Classes in Scala
Demonstrating Case Classes in ScalaDemonstrating Case Classes in Scala
Demonstrating Case Classes in ScalaBoldRadius Solutions
 
Functional Programming - Worth the Effort
Functional Programming - Worth the EffortFunctional Programming - Worth the Effort
Functional Programming - Worth the EffortBoldRadius Solutions
 

Mehr von BoldRadius Solutions (18)

Introduction to the Typesafe Reactive Platform
Introduction to the Typesafe Reactive PlatformIntroduction to the Typesafe Reactive Platform
Introduction to the Typesafe Reactive Platform
 
Partial Functions in Scala
Partial Functions in ScalaPartial Functions in Scala
Partial Functions in Scala
 
Towards Reliable Lookups - Scala By The Bay
Towards Reliable Lookups - Scala By The BayTowards Reliable Lookups - Scala By The Bay
Towards Reliable Lookups - Scala By The Bay
 
How You Convince Your Manager To Adopt Scala.js in Production
How You Convince Your Manager To Adopt Scala.js in ProductionHow You Convince Your Manager To Adopt Scala.js in Production
How You Convince Your Manager To Adopt Scala.js in Production
 
Why Not Make the Transition from Java to Scala?
Why Not Make the Transition from Java to Scala?Why Not Make the Transition from Java to Scala?
Why Not Make the Transition from Java to Scala?
 
String Interpolation in Scala | BoldRadius
String Interpolation in Scala | BoldRadiusString Interpolation in Scala | BoldRadius
String Interpolation in Scala | BoldRadius
 
Value Classes in Scala | BoldRadius
Value Classes in Scala | BoldRadiusValue Classes in Scala | BoldRadius
Value Classes in Scala | BoldRadius
 
Scala Days Highlights | BoldRadius
Scala Days Highlights | BoldRadiusScala Days Highlights | BoldRadius
Scala Days Highlights | BoldRadius
 
What Are For Expressions in Scala?
What Are For Expressions in Scala?What Are For Expressions in Scala?
What Are For Expressions in Scala?
 
Domain Driven Design Through Onion Architecture
Domain Driven Design Through Onion ArchitectureDomain Driven Design Through Onion Architecture
Domain Driven Design Through Onion Architecture
 
Pattern Matching in Scala
Pattern Matching in ScalaPattern Matching in Scala
Pattern Matching in Scala
 
What are Sealed Classes in Scala?
What are Sealed Classes in Scala?What are Sealed Classes in Scala?
What are Sealed Classes in Scala?
 
Scala: Collections API
Scala: Collections APIScala: Collections API
Scala: Collections API
 
Immutability in Scala
Immutability in ScalaImmutability in Scala
Immutability in Scala
 
Scala Days 2014: Pitching Typesafe
Scala Days 2014: Pitching TypesafeScala Days 2014: Pitching Typesafe
Scala Days 2014: Pitching Typesafe
 
Code Brevity in Scala
Code Brevity in ScalaCode Brevity in Scala
Code Brevity in Scala
 
Demonstrating Case Classes in Scala
Demonstrating Case Classes in ScalaDemonstrating Case Classes in Scala
Demonstrating Case Classes in Scala
 
Functional Programming - Worth the Effort
Functional Programming - Worth the EffortFunctional Programming - Worth the Effort
Functional Programming - Worth the Effort
 

Kürzlich hochgeladen

JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024Mind IT Systems
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptkinjal48
 

Kürzlich hochgeladen (20)

JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.ppt
 

Easy Scalability with Akka - Scala Days 2015

  • 1. Easy Scalability with Akka Distribute your domain
  • 2. Who? ● BoldRadius Solutions ○ boldradius.com ○ Typesafe Partner ○ Scala, Akka and Play specialists ○ Ottawa, Saskatoon, San Francisco, Boston, Chicago, Montreal, New York, Toronto ● Michael Nash, VP Capabilities ○ @MichaelPNash ○ michael.nash@boldradius.com
  • 3. What? What will I know in the next 45 minutes? ● Distributed Domain-Driven Design ○ What is it, and how does it apply to project? ○ What is CQRS and ES, and how to they relate? ● Akka ○ How to do DDDD and CQRS/ES with Akka ○ What is different in a DDDD Akka project, and why? ○ How does this kind of project scaled compared to a “normal” project, even with Akka?
  • 4. Scalability ● Scalability is not performance ○ Performance increasing reduces response time ○ Scalability increasing improves capacity in relation to added resources ● Scalability can be... ○ Vertical (Scaling “up”): Bigger box (or boxes) ○ Horizontal (Scaling “out”): More boxes
  • 5. ● The Actor model for the JVM ○ A share-nothing distributed message- driven compute model ○ Scala and Java APIs ● Like Erlang/Elixir processes, only better ○ Distributed by design, local is an optimization
  • 6. CQRS and ES ● Command Query Responsibility Segregation ○ Separate the read and write paths of the problem ● Event-Sourcing ○ State is built from a journal of events ● One doesn’t need the other, but they can be combined very effectively
  • 9. Domain ● Your application’s domain is the part that relates to the problem ● The rest of your application is infrastructure, basically ● If you have a Payroll app, then Employees, Benefits, Pay Stubs are in your domain
  • 10. DDDD ● Domain-Driven Design (DDD) is not only for Object-Oriented Systems ● Object-Oriented and Functional are not an either-or choice, or at odds with each other ● Distributed Domain Driven Design is a natural fit with the actor model
  • 12. Non-Distributed Domain ● Basic CRUD (Create, Read, Update, Delete) ● Writes and reads to same database ○ Always consistent ● Scaled via multiple identical replicas ○ and load balancing on HTTP ● Bottlenecks on contention ○ reads and writes interfere with each other
  • 13. When we scale the traditional model… ● We add multiple servers ○ so there’s more than one copy of each domain instance (potentially) ● Each must read state every time ○ from the shared resource (the database) ● Each must write (fully) to the same shared resource to avoid conflict ● Scalability is limited
  • 15. With Akka and DDDD ● Wouldn’t it be great if you could just keep your domain instances in memory? ● But how to recover from its volatile nature: an event journal!
  • 17. But I have too many instances! ● Only active domain instances are in memory ● Instances activate and passivate as needed
  • 19. Ingredients ● Akka - Scala API ○ with Clustering and Persistence modules ● Cassandra ○ And the DataStax driver ● Typesafe Activator ○ For our template
  • 20. What’s Different? ● Domain object instances are transient, and memory-resident when in use ● Reads and writes don’t contend for resources ● Cluster can be grown until domain instances per node = 1
  • 21. What do you get? ● Scalability ○ Add nodes to handle higher load on the fly ● Failover ○ No single point of failure or contention ○ Instances can be created anywhere as needed ● Simplicity ○ Build your app without having to worry about scalability later
  • 22. Consider your own domain ● Consider writes independently from reads ○ Are they really the same even now? ● Deployment structure is critical to get all benefits safely
  • 23. Example Applications Online Auctions: Two implementations ● Users place bids for online auctions ● Most code shared ● Identical APIs ● Identical deploy structure, same machines, same number of nodes ● One CRUD, One CQRS/DDDD
  • 24. CRUD Actor ● Uses Akka ● Receives request ● Writes to datastore directly
  • 26. CQRS/DDDD Actor ● In-memory state ● Recovers state from journal on startup ● Uses become to change states ● Passivates on timeout
  • 30. Test Setup 3 Cassandra Nodes (Clustered) 3 Akka processing Nodes (Clustered) 1 Front-end (Part of the Akka cluster) All Small AWS Instances 100 simultaneous users 100 auctions Two minutes of bidding
  • 32. Response Time Distribution CRUD CQRS Scale is different!
  • 34. Response Time Percentiles: CQRS Note that the scale is different!
  • 36. Requests per Second: CQRS Scale is different!
  • 37. Total Requests/Responses in 2 mins CRUD: 75,991 CQRS: 199,357
  • 38. Adding more nodes... ● CRUD version ○ Contention increases ○ Diminishing benefit to adding more nodes ○ Tuning doesn’t help much ● CQRS version ○ No contention ○ Near-linear benefit until number of nodes = number of auctions ○ Tuning is very important (type of nodes too…)