Architectural Overview of Distributed Parcel Tracking and Management System. Presented at Melbourne Alt.NET, 2015.

1. Architectural Overview of
Distributed ParcelTracking
and Management System
Mark Cheeseman
mcheeseman@visaglobal.com.au
http://hombredequeso.id.au/

2. Public Health Warning (Required by Law…)
The following presentation includes graphic scenes of UI torture and
SQL Server abuse that may disturb some viewers.
Usable User Interface design and relational use of relational databases
were both harmed in the making of this product.
Viewers who suspect they may find such scenes disturbing may wish
to confirm the presence of a sick bag in the back of the seat before
you.

3. Website/ API
Business Services
Process Bookings
Produce Labels
Customs
Billing
Parcel Scan In/ Out
...
Tracking
Website
Customs
Customers
Parcel
Recipients
AusPost
NZ Post
Whip
Operators
Conceptual Diagram
...

4. Website API *
Tracking
Website *
API Update Service
Tracking Update
Service
Parcel Audit
Service *
AusPost Service *
Billing Service *
Booking Service *
Cyberfreight
Service *
MAWB Service
NZ Post Service * NZ Post API Service
Reporting Service *
Scan Processing
Service*
Email Processing
Service *
File Transfer
Service
Customers/
Operators
(Web Browser)
Scan/Print
(Application)
Parcel Recipients
(Web Browser)
Cyberfreight
(Application)
NZ Post (API)
Cyberfreight
Customs (SMTP)
AusPost (SFTP)
Logical Diagram
Bus
* = has own database

5. Azure VM
Azure
Service Bus
NSB Message
Transport
Physical Diagram
Azure SQL
Server
· API
· Booking
· AusPost
· Cyberfreight
· ...
Azure SQL
Server
· Tracking Website
Azure Web App
Website/API
(ASP.NET MVC API)
Azure Web App
Tracking Website
(ASP.NET MVC API)
RavenDb
Server
· Booking NSB EP
· AusPost NSB EP
· NZPost NSB EP
· ...
Service: Booking NSB Endpoint
Service: AusPost NSB Endpoint
Service: NZPost NSB Endpoint
Service: Cyberfreight NSB Endpoint
...

11. REST API: Siren
Specification:
https://github.com/kevinswiber/siren

12. {
"class": ["booking"],
"properties": {
"bookingNumber": "BREDH10000910",
"merchant": "Red Hare",
"merchantCode": "REDH",
"parcelCount": 1000,
"created": "2015-05-24T11:47:37.4721121Z",
"processes": "Labelling, Customs",
"parcelsRecordedForCustomsCount": 1000,
"parcelsRecordedForCustomsTimestamp": "2015-05-24T11:47:41.2653291Z",
"ausPostLabelsGenerated": "2015-05-24T11:47:48.9757701Z",
"ausPostLabelsRecordedForCustomsCount": 1000,
"ausPostLabelsRecordedForCustomsTimestamp": "2015-05-24T11:48:00.2774165Z",
"labelledSpreadsheetCreated": "2015-05-24T11:48:02.6825541Z"
},
"entities": [
{
"class": ["bookingSubmission"],
"rel": ["/rel/booking-bookingsubmission"],
"links": [
{
"rel": ["self"],
"href": "http://localhost:54747/api/Merchant/BookingSubmissions/92be90a5-710d-40ef-bb6c-a4a201672282"
}
]
}
],
"links": [
{
"rel": ["self"],
"href": "http://localhost:54747/api/Merchant/Bookings/BREDH10000910"
},
{
"rel": ["file","xlsx","booking/labelledContent"],
"href": "http://localhost:54747/api/Merchant/Bookings/BREDH10000910/labelledContent"
}
],
"actions": [
{
"name": "post-bookingDespatchManifestRequest",
"title": "Generate Despatch Manifest For Booking",
"method": "POST",
"href": "http://localhost:54747/api/Merchant/Bookings/BREDH10000910/bookingDespatchManifestRequests",
"type": "application/x-www-form-urlencoded",
"fields": []
}
]
}
GET http://localhost:54747/api
/Merchant/Bookings/BREDH10000910

13. {
"class": ["booking"],
"properties": {
"bookingNumber": "BREDH10000910",
"merchant": "Red Hare",
"merchantCode": "REDH",
"parcelCount": 1000,
"created": "2015-05-24T11:47:37.4721121Z",
"processes": "Labelling, Customs",
"parcelsRecordedForCustomsCount": 1000,
"parcelsRecordedForCustomsTimestamp": "2015-05-24T11:47:41.2653291Z",
"ausPostLabelsGenerated": "2015-05-24T11:47:48.9757701Z",
"ausPostLabelsRecordedForCustomsCount": 1000,
"ausPostLabelsRecordedForCustomsTimestamp": "2015-05-24T11:48:00.2774165Z",
"labelledSpreadsheetCreated": "2015-05-24T11:48:02.6825541Z"
},
"entities": [
{
"class": ["bookingSubmission"],
"rel": ["/rel/booking-bookingsubmission"],
"links": [
{
"rel": ["self"],
"href": "http://localhost:54747/api/Merchant/BookingSubmissions/92...82"
LogicalView

14. ]
}
],
"links": [
{
"rel": ["self"],
"href": "http://localhost:54747/api/Merchant/Bookings/BREDH10000910"
},
{
"rel": ["file","xlsx","booking/labelledContent"],
"href": "http://localhost:54747/api/Merchant/Bookings
/BREDH10000910/labelledContent"
}
],
"actions": [
{
"name": "post-bookingDespatchManifestRequest",
"title": "Generate Despatch Manifest For Booking",
"method": "POST",
"href": "http://localhost:54747/api/Merchant/Bookings
/BREDH10000910/bookingDespatchManifestRequests",
"type": "application/x-www-form-urlencoded",
"fields": []
}
]

15. API Event Store
Bus (NSB)
API Update
Service
WEB API
Event Stream to Siren
Entity Transform
Event
Store
Http Request (GET)
NSB Events from various endpoints
(Booking, AusPost, Customs)
“Event Store”
Events
NSB Events
Siren Entity

16. Business Processes Involving
Multiple Services
(Long Running Business Processes)

17. ProcessManager: Saga asProcessController
Booking
Endpoint*
Saga
AusPost
Endpoint*
Customs
Endpoint*
7. Booking “fully”
accepted
Saga
State
Fastway
Endpoint*
1. BookingReceivedEvent
2. CreateApLabelsCommand
3. CreateFwLabelsCommand
6. CreateConsolidations
Command
5. ApLabelsCreatedEvent
4. FwLabelsCreatedEvent

18. ProcessManager: Routing Slip
Booking
Endpoint*
AusPost
Endpoint*
Customs
Endpoint*
Fastway
Endpoint*
ProcessBooking
SubmissionCommand
Routing Slip (on Booking Message):
1. AusPostEndpoint
2. FastwayEndpoint
3. CustomsEndpoint
Booking Booking Booking
Booking
(“fully” accepted)

19. ProcessManager: Event Driven Booking Processeson Event
Booking
Endpoint*
AusPost
Endpoint*
Customs
Endpoint*
Fastway
Endpoint*
1. BookingCreatedEvent
2. AusPostLabelsCreatedEvent
3. FastwayLabelsCreatedEvent
Booking Representation on all Events:
{
Guid BookingId
BookingProcess[] Processes
}
public enum BookingProcess
{
Labelling = 1,
Customs = 2,
AusPostPreLabelled = 3
}
4. Booking “fully”
accepted

20. Solving Big(ish) Data on the Bus
AusPost Service *
Booking Service *
Cyberfreight
Service *
NZ Post Service * NZ Post API Service
Email Processing
Service *
File Transfer
Service
The IdealisticVision...
(Logical) Bus

21. AusPost Service *
Booking Service *
Cyberfreight
Service *
NZ Post Service * NZ Post API Service
Email Processing
Service *
File Transfer
Service
Bus: Message Bus+ Data Bus
OHS
ACL
ACL
ACL
OHS
OHS
ACL
ACL
(Message) Bus(Data) Bus

22. AusPost Service *
Booking Service *
Cyberfreight
Service *
NZ Post Service * NZ Post API Service
Email Processing
Service *
File Transfer
Service
Data BusImplementation
SQL
Database
NSB (Physical) Transport:
Azure Service Bus
Data Bus: Implemention via
SQL queries

23. Not AllTransports are Created Equal…

24. References
Siren Specification
https://github.com/kevinswiber/siren
Richardson Maturity Model (for API's)
http://martinfowler.com/articles/richardsonMaturityModel.html
Recommended books on REST
Richardson and Amundsen, RESTfulWeb APIs.
Webber et. al., REST in Practice: Hypermedia and Systems Architecture

25. Messaging in General
Hohpe andWoolf, Enterprise Integration Patterns
(for various forms of routing as process management see ch. 7)
Blog Series on NServiceBus and Process Management Patterns
https://lostechies.com/jimmybogard/2013/05/14/saga-patterns-wrap-up/
(Top of post has links to all post in series)
https://lostechies.com/jimmybogard/2014/10/02/nservicebus-5-0-behaviors-in-action-routing-
slips/
Event Based Systems
VaughnVernon, Implementing Domain-Driven Design, ch. 8.
Dynamic Event Replay Mechanism
http://blogs.msdn.com/b/davidebb/archive/2010/01/18/use-c-4-0-dynamic-to-drastically-
simplify-your-private-reflection-code.aspx