Slides from my talk at QCon London on 9th of April 2024

1. @berndruecker
Are you done yet?
Mastering
long-running processes
in modern architectures

2. Let‘s talk about food

3. How does ordering Pizza work?
Pizza
Place
You
Phone Call
Synchronous blocking communication
Feedback loop (ack, confirmation or rejection)
Temporal coupling (e.g. busy, not answering)
Pizza
Place
You
Email
Asynchronous non-blocking communication
No temporal coupling
Pizza
Place
You
A feedback loop might make sense
(ack, confirmation or rejection)
Email
Confirmation Email
@berndruecker

4. Feedback loop != result
Pizza
Place
You
Email
Confirmation Email
Pizza Delivery
Feedback (ACK, confirmation, rejection)
Result
@berndruecker

5. Only the first communication step is synchronous / blocking
Pizza
Place
You
PUT/order
HTTP 200
Pizza Delivery
The task of
Pizza making is
long running
@berndruecker

6. Synchronous blocking behavior for the result?
Bad user experience
Does not scale well
@berndruecker

7. Scalable Coffee Making
https://www.enterpriseintegrationpatterns.com/ramblings/18_starbucks.html
Photo by John Ingle
@berndruecker

8. Long running

9. Long running

10. Long running basically means waiting

11. When do services want to wait? Some business reasons…
Human work Waiting for response
Let some time pass

12. Why is waiting a pain?
Persistent state How to solve the technical challenges without adding accidential
complexity?
Monitoring &
Operations
Visibility
Versioning
Scheduling &
Timeouts
Domain Logic
Scalability &
Resilience

13. Warning:
Contains Opinion

14. bernd.ruecker@camunda.com
@berndruecker
http://berndruecker.io/
Bernd Ruecker
Co-founder and
Chief Technologist of
Camunda

15. Workflow Engine aka Process Engine aka Orchestration Engine

16. Live hacking
https://github.com/berndruecker/customer-onboarding-camunda-8-springboot

17. Customer Unboarding
https://github.com/berndruecker/customer-onboarding-camunda-8-springboot

18. @berndruecker

20. There are also technical reasons
why services need to wait…

21. When do services need to wait? Some technical reasons…
Asynchronous communication
Wait for
responses
Wait for
availability
Especially failure scenarios
Unavailability of peers

22. Live hacking
https://github.com/berndruecker/customer-onboarding-camunda-8-springboot

23. Photo by Tookapic, available under Creative Commons CC0 1.0 license.

24. „There was an error
while sending your
boarding pass“

25. Check-in
Web-UI
Me
Current situation

26. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation

27. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation

28. Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Failure will happen.
Accept it!
But keep it local!
Be resilient.

29. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part

30. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part

31. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part
Stateful
Retry

33. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Possible situation – much better!

34. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Possible situation – much better!
Stateful
Retry

35. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Stateful
Retry
Possible situation – much better!
The failure
never leaves
this scope!

36. „But the customer wants a synchronous response!“
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
„Eh – no!“

37. Let‘s add some money…
@berndruecker
Booking Payment
Credit
Card

38. Payment
Stateful retry by using an orchestration engine
Credit
Card
REST

39. Distributed systems introduce more complexity
Credit
Card
Payment
REST

40. It is impossible to
differentiate certain
failure scenarios.
Independant of
communication style!
Service
Provider
Client

41. Distributed systems introduce more complexity
Credit
Card
Payment
REST

42. Distributed systems introduce more complexity
Credit
Card
Payment
REST
Cancel
charge
Using an orchestration engine allows to do long
running processes reliably, and end-to-end

43. retrievePayment
HTTP 200 OK
HTTP 202 ACCEPTED
Payment
Happy case: Synchronous response
Otherwise: asynchronous
Embrace asynchronicity

44. This allows to extend payment Options
@berndruecker

45. Further requirements when extending payment Options
begin commit
{local TX}
Customer Credit Service
begin commit
{local TX}
Credit Card Service
@berndruecker

46. Distributed transactions using compensation *
Compensation
@berndruecker

47. If you want to
dive into code…
https://github.com/berndruecker/flowing-retail

48. Long running capabilities
are essential to design
good service boundaries
(= a good architecture)
@berndruecker

49. Example
Booking Payment
Retrieve
Payment
@berndruecker

50. Example
Booking Payment
Credit
Card
Retrieve
Payment
@berndruecker

51. Example
Booking Payment
Credit
Card
Retrieve
Payment
Rejected
@berndruecker

52. Example
Booking Payment
If the credit
card was
rejected, the
customer can
provide new
details
Credit
Card
Retrieve
Payment
Rejected
Rejected
@berndruecker
???

53. Payment
failed
Who is responsible to deal with problems?
Booking Payment
If the credit
card was
rejected, the
customer can
provide new
details
Credit
Card
Retrieve
Payment
Rejected
Payment
received
@berndruecker

54. Payment
failed
(Potentially) long running services
Booking Payment
Credit
Card
Retrieve
Payment
Rejected
Payment
received
@berndruecker

55. Long running services
Booking Payment
Credit
Card
@berndruecker

56. Orchestration != monolithic processes
Booking Payment
Credit
Card
@berndruecker

57. Long running avoids „god services“
Booking Payment
If the credit
card was
rejected, the
customer can
provide new
details
Credit
Card
Retrieve
Payment
Rejected
Rejected
@berndruecker
A few
smart god services
tell
anemic CRUD services
what to do
Sam Newmann

58. Long running capabilities
are essential to design
good service boundaries
(= a good architecture)
@berndruecker

59. Being able to implement
long running services
makes it easy to distribute
responsibilities correctly
@berndruecker

60. Being able to implement
long running services
makes it easy to embrace
async/non-blocking
@berndruecker

61. You need a
process orchestration capability
that is available as-a-service
within your organization
to create competitive architectures

62. What we see most successful: Center of Excellence
https://camunda.com/process-orchestration/automation-center-of-excellence/

63. Focus: Enablement and platform
Business Unit
or Domain
Business Unit
or Domain
Business Unit
or Domain
Process Automation
Center of Excellence
Solution
Delivery
Solution
Delivery
Solution
Delivery
Solution
Delivery
Solution
Delivery
enable

64. But isn’t a central CoE
harming
team autonomy?
(This is a rhetorical question - the answer is of course NO)

65. @berndruecker

66. Center of
Excellence
Domain

67. Autonomy of teams?

68. Golden Paths
https://engineering.atspotify.com/2020/08/how-we-use-golden-paths-to-solve-fragmentation-in-our-software-ecosystem/
“We found that
rumour-driven development
simply wasn’t scalable”
@berndruecker

69. https://backstage.io/ (OSS, Made with at Spotify) @berndruecker

70. At Twilio, we do it by offering what we call the paved path.
These are mature services that you can just pull off the shelf,
adopt, and get up and running super quickly.
But if a team wants to diverge from that path they can make
that decision. You just still have to pass the same rigorous bar
of quality, reliability, and security.
So what you do is you create the incentive structure for teams
to take the paved path. It’s a lot easier. But if they really have
to go a different route, you make it possible for them to do
that.

71. Graphical models?
@berndruecker

72. BPMN
Business Process
Model and Notation
ISO Standard
@berndruecker

73. Living documentation for long-running behaviour
@berndruecker

74. Living documentation for long-running behaviour
@berndruecker

75. BizDevOps
@berndruecker

76. Decisions about
long running behavior
need to be elevated
to the business level

77. You need to redesign
your customer journeys
to leverage your
hipster architecture
Shutterstock

87. Yeah - everything
asynchronously :-)

88. Recap
• You need capabilities for long running behavior for technical and
business reasons
• Process orchestration platforms are a great fit
• This allows you to design better service boundaries, implement quicker
with less accidental complexity, embrace asynchronoucity and provide
a better customer experience (plus increase operational efficiency and
reduces risk)
• Organize central enablement (center of excellence, platform as-a-
service) for easy adoption at scale
@berndruecker

89. Want to learn more?
http://camunda.com/

91. https://www.berndruecker.io/ | bernd.ruecker@camunda.com