NServiceBus is a lightweight messaging framework for .NET that helps make distributed systems more reliable, scalable, and extensible using a message-oriented architecture. It provides transaction management, automatic retries on failure, and supports various queues including MSMQ. Messages are sent asynchronously in the form of commands or events. Commands are processed by a single endpoint while events can be published to multiple subscribers. This improves reliability over a traditional request/response model. The document discusses how NServiceBus can be used to address issues like transaction management and data consistency when processing orders across multiple services in a distributed system.
2. What is NServiceBus?
Lightweight messaging framework for designing distributed systems in
.NET
An Enterprise Service Bus
Helps to make distributed systems more
Reliable
Scalable
Extensible
Founded in 2006 by UDI Dahan, an international renowned expert on
software architecture and design
Services and support provided by Particular Software
Open Source but not free
Source code available at https://github.com/Particular/NServiceBus
Binaries available as NuGet packages
3. Remote Procedure Call (Request/Response)
Request: Order
Website
Response: Order ID
Order Service
PlaceOrder(order)
Synchronous – The website is blocked while calling the Order Service
Messaged Oriented Architecture (One-way messaging)
Message: PlaceOrder
Website
Message Queue
Connect
Receive
PlaceOrder message
Windows Service
Process the PlaceOrder message
Asynchronous – The website is NOT blocked while the place order is processed
4. NServiceBus Terminology & Components
Sender
/Send-Only
Endpoint
Message,
Command
or Event
Message: PlaceOrder
Website
Message Queue
Connect
Receive
PlaceOrder message
Windows Service
Process the PlaceOrder message
Message Handler
Worker
Endpoint,
Host or
Receiver
5. Messages in NServiceBus
Just simple .NET classes with properties
2 types of messages
Command
• Do something: PlaceOrder
• Sent from one or more senders, processed by one endpoint
• Implement ICommand
Event
• Something has happened: OrderWasPlaced
• Published from one sender, processed by several endpoints/subscribers
• Implement IEvent
6. Command
Strongly coupled to
the receiving
endpoint
Command: PlaceOrder
Website
Command: PlaceOrder
Message Queue
Endpoint
Event (Publish/Subscribe)
Subscribing
Endpoint
Loosely coupled to
the receiving
endpoints
Event: OrderWasPlaced
Worker
NServiceBus will add 3
messages to the
queue, one for each
subscriber
Event: OrderWasPlaced
Message Queue
Subscribing
Endpoint
Subscribing
Endpoint
7. Queues
First in-first out datastructure for storing and retrieving messages
PlaceOrder message 4
First-In
Queue
PlaceOrder message 3
PlaceOrder message 2
PlaceOrder message 1
First-Out
Default queue in NServiceBus is Microsoft Message Queuing (MSMQ)
Available on all Windows editions since NT 4 and Windows 95
Also support for
ActiveMQ
RabbitMQ
SQL Server
Windows Azure Queues
Windows Azure ServiceBus
One endpoint has one queue, Send-Only endspoints do not have queues
8. How to Send & Handle Messages
Command: PlaceOrder
Website
Send
Command: PlaceOrder
Message Queue
Endpoint
Handle
(Command or Event)
9. Summary
Sender endpoint
Sends a message
Can be a website, desktop application or even a worker endpoint
Command
Message where we want to do something concrete
ICommand interface
Event
Message where we notify about something that has happened
IEvent interface
Message handler
Processes a given message
IHandleMessages<Message> interface
Worker endpoint
Finds and executes the handler for the message
Can be a Windows Service, console application, WCF, website
10. NServiceBus Requirements
MSMQ (default) or any other supported queue
ActiveMQ
RabbitMQ
SQL Server
Windows Azure Queues
Windows Azure ServiceBus
Distributed Transaction Cordinator (DTC)
RavenDB (default) or a relational database
Download the installer from NServiceBus from http://particular.net/ to
setup the required components
11. NuGet packages
NServiceBus Interfaces
Interfaces that we need to define messages
NServiceBus
Most of the code that drives NServiceBus except hosting
Use for send-only endpoints
NServiceBus Host
Console application that can be installed as a Windows Service
Used to process/handle messages
NServiceBus Testing
Framework for testing NServiceBus endpoints
12. The Case – The eBook Shop
1. Create the order in the
database
2. Auto-pay the order using a
previously stored credit
card
3. Send an order confirmation
by e-mail
4. Deliver the book to the
Windows Phone device
13. The Usual Way - Request/Response
Place order
1. Add order
Browser
OrderController
Confirmation page
with order ID
Database
2. Charge credit card
PaymentService
3. Send mail
SMTP-Server
4. Deliver ebook
DeviceService
14. Problem #1 – The order is lost during an error
Place order
1. Add order
Browser
OrderController
Database
Error page
Exception
2. Charge credit card
Error
PaymentService
3. Send mail
SMTP-Server
4. Deliver ebook
DeviceService
Conseqence
The database is rolled back
User receives an error
Order must be sent again
15. Problem #2 – Poor Transaction Management
Place order
1. Add order
Browser
OrderController
Database
Error page
2. Charge credit card
PaymentService
3. Send mail
SMTP-Server
Exception
4. Deliver ebook
Error
DeviceService
Conseqence:
The credit card is charged (no transaction support)
A mail saying the order was successful was sent (no transaction support)
The order do not exist in the database (supports transaction – rolled back)
The user will receive an error
16. Let’s Fix This With NServiceBus
Change to a message oriented architecture
Add PlaceOrder
message
Place order
Browser
Queue
OrderController
Confirmation page
Old
New
17. The Message Must Also Be Processed
MSMQ
1. Connect
2. Receive message PlaceOrder
Worker
(Windows Service)
PlaceOrder
3. PlaceOrder
Message Handler
4. Add order
Database
5. Charge credit card
PaymentService
6. Send mail
SMTP-Server
7. Deliver ebook
DeviceService
18. When an Error Occur
4. Rollback. Put the message
back on the queue
and retry
MSMQ
1. Connect
2. Receive message
PlaceOrder
Worker
(Windows Service)
PlaceOrder
3. PlaceOrder
Error
Message Handler
19. More about MSMQ
Max 4MB message size
Persists messages to files. Default location C:WINDOWSSystem32msmqstorage
Guranteed once-and-only-once delivery
The message will only be handled once
Supports transactional queues and distributed transactions (using DTC)
If an error occur both changes to the database and to the queue can be rolled back
Supports store-and-forward
MSMQ will store messages locally before forwarding it to the queue on another server
MSMQ will confirm when the message has been written to the disk in the outgoing
queue, not when it has been delivered to the server
Important feature for distributed systems with unrealiable networks
Server A
Sender
MSMQ
Outgoing
queue
Server B
Message
MSMQ
Input
queue
Worker
20. How NServiceBus Handles Failure
Transaction management and rollback are automatically handled by
NServiceBus in message handlers
When an error occur
1. The message is put back on the queue
2. Attempt to process the message 5 times, wait for 10 seconds
3. Attempt to process the message 5 times, wait for 20 seconds
4. Attempt to process the message 5 times, wait for 30 seconds
5. Attempt to process the message 5 times, send the message to an error queue
The time interval is configurable
Messages can be moved back to the original queue for replay using
ReturnToSourceQueue.exe tool.
21. A challenge – The Order ID
Old
New
The order ID has not been generated yet – it’s generated in the message
handler at the worker endpoint
22. Possible Solutions
Change the UI to not show the order ID
Use a GUID (Guid.NewGuid()) or find another way to generate the ID in the controller
instead of in the database
• message.OrderId = Guid.NewGuid();
Bus.Send(message);
ViewBag.OrderId = message.OrderId;
• 5773DD0E-0AB0-446B-8649-B2D43D7DA4AA is not a very user friendly ID
Move OrderRepository.Add(order) to the Controller:
• var orderId = OrderRepository.Add(order);
Bus.Send(message);
• Less reliable and more error prone solution
Use Send/Reply to simulate request/response
• The controller will wait for a PlaceOrderCompleted message that the PlaceOrder handler will send
• Too much overhead
• An Anti-pattern
23. We Still Have Poor Transaction Management
1. Add order
Handle PlaceOrder
Database
2. Charge credit card
PaymentService
3. Send mail
SMTP-Server
4. Deliver ebook
Exception
Error
DeviceService
Conseqence:
The credit card is charged (no transaction support)
A mail saying the order was successful was sent (no transaction support)
The order do not exist in the database (supports transaction – rolledback)
The message is put back on the queue and will be retried = credit card will be charged
twice
24. Solution
Split into man small messages - One message per transactional boundary
Place order
OrderController
Browser
1. Add PlaceOrder message
PlaceOrder message
2. Handle PlaceOrder
Add PayOrder message
If an error occur only
DeliverEbook is affected. The
message will be put back on the
queue and retried
SendMail message
4. Handle SendMail
DeliverEbook message
5. Handle DeliverEbook
PayOrder message
Queue
Add SendMail &
DeliverEbook message
3. Handle PayOrder
28. Yet another transactional issue
MSMQ
Processed
ok
Handle PayOrder message
Charge credit card
Request
Handle PayOrder
Request: Error
PaymentService
Payment OK
Network
error
External
Payment
Gateway
The message will be put back on the queue and processed again
Consequence: The credit card will be charged more than once
29. Solution
Operations should be idempotent
Idempotent operations can be run many times without changing the result
MSMQ
Handle PayOrder message
Charge credit card
Request
Handle PayOrder
Payment OK
PaymentService
Has the order
been paid?
Use OrderId
Database
Payment OK
External
Payment
Gateway
Has the
message
already been
processed?
Use order ID
or a GUID
Database
30. Another Challenge – Eventual Consistency
Users clicks on View
Order History
The PlaceOrder message is processed async by another process
If it has not gotten processed yet then the new order will not be in the list
Solution
Some UI trick?
Move the OrderRepository.Add to the controller
Make sure the messages are handled fast enough
• Define a SLA on the message to help monitoring
31. Publish/Subscribe
Place order
OrderController
Browser
1. Add PlaceOrder message
PlaceOrder command
2. Handle PlaceOrder
Add PayOrder command
With Publish/Subscribe the
Subscribers are loosely coupled
to the publisher
Subscribe to OrderWasPaid
4. Handle SendMail
Subscribes to OrderWasPaid
Queue
3. Handle PayOrder
Add OrderWasPaid event
OrderWasPaid
Subscribes to OrderWasPaid
4. Handle DeliverEbook
PayOrder command
OrderWasPaid
NServiceBus will add 2
messages to the
queue, one for each
subscriber
33. Scaling
Scale up
Easy to scale up if business processes are split into multiple messages
Increase the number of threads in the config – NServiceBus will concurrently process
multiple messages
Scale out
Use the Distributor or the Master (Both a distributor and a worker)
Worker #1
Message
MSMQ
Distributor
Worker #2
Worker #3
The distributor will forward the message to a worker that is ready to process the message
34. How the Distributor Works
1. Worker:
Send I’m ready
message
I’m ready message
Distributor:
Control Queue
I’m ready message
Distributor
Any messages in
the input queue?
PlaceOrder message
1. Website:
Send
PlaceOrder
message
Send PlaceOrder
message from input
queue
PlaceOrder message
Distributor
Input Queue
Yes
No
PlaceOrder message
Distributor
Find worker
through
message in
Storage
queue
Store I’m ready
message
Storage Queue
35. Other Features
Sagas
Long running workflow like processes
State is shared between multiple message handlers
Scheduling
Send a message every x minute. A handler will handle the message
Unobtrusive mode
Use your own ICommand, IEvent etc. to reduce coupling to NServiceBus
Supports
Various dependency injection containers
Logging frameworks
Etc.
++
36. NServiceBus License & Pricing
Open source
Used to be free, but not anymore
Indefinitely license with 1 year maintenance updates
• $500 USD per processing core
• $250 USD per sending core
Monthly subscription
• $35 USD per processing core
• $17 USD per sending core
Development and testing environments + disaster recovery and passive
backups are free
Developers need a license (free) that must be renewed every 3rd month
through particular.net