Odoo Experience 2020 https://youtu.be/CLdeQLxwJPU

2. How to design
Resilient
Odoo Crons?

3. 1.Introduction

4. Resilient
able to return quickly to a
previous good condition after
problems

5. Use case 1
Create an invoice for subscription
every month

6. Use case 2
Synchronize contacts with an
external tool every day

7. 2. Pitfalls

8. Use case 1
Naive implementation
➔ Lack of robustness: One fails, all fail
➔ Monolithic: In case of timeout
everything is rolled back
● Need to process all data again
➔ Timeout (or rollback) end up with
inconsistent state if emails are sent
➔ If an invoice is skipped one month, it
will never invoice that month
➔ High risk of concurrent update: long
transaction updating data
➔ Run twice the same month, end up
with 2 invoices
# defined on sale.subscription
# Run Once a month
def _run_subscription_invoice(self):
subscriptions_to_invoice = self.search([
('stage_id.category', '=', 'progress'),
])
for subscription in subscriptions_to_invoice:
subscription.create_invoice()

9. Use case 2
Naive implementation
➔ Always process all the data
● Data to process will only grow
● It will timeout
➔ Monolithic: In case of timeout the
process will retry forever
➔ If it runs twice in the same day, it will
synchronize twice
➔ Run only once a day, if service is
unavailable at that specific time: no
synchronization that day
# Defined on res.partner
# Run Once a day
def _sync_partner(self):
partners = self.search([])
data = partners._prepare_data()
self._send_data(data)

10. What’s so bad
about timeout?

11. ● It’s impossible to handle the
exception
● Next execution date is not updated.
It can run forever

12. ● It’s impossible to handle the
exception
● Next execution date is not updated.
It can run forever

13. 3. Features

14. Idempotent
# Use case 1
# Run Once a DAY
def _run_subscription_invoice(self):
subscriptions_to_invoice = self.search([
('stage_id.category', '=', 'progress'),
('next_invoice_date', '=', today),
])
for subscription in subscriptions_to_invoice:
subscription.create_invoice()
subscription.next_invoice_date +=
relativedelta(month=1)
# Use case 2
# Run whenever you want
def _sync_partner(self):
# Dirty is set to True when the partner is modified
partners = self.search([('dirty', '=', True)])
data = partners._prepare_data()
self._send_data(data)
partners.write({'dirty': False})
An operation that can be applied multiple
times without changing the result beyond
the initial application.
➔ Cron(rec) ~ Cron(rec); Cron(rec); ...
➔ Make sure that data are only
processed once
➔ Keep track of processed records
+ Frequencies of cron and data
processing don’t need to be correlated
+ It’s safe to retry

15. Incremental
# Use case 1
def _run_subscription_invoice(self):
subscriptions_to_invoice = self.search([
('stage_id.category', '=', 'progress'),
('next_invoice_date', '=', today),
])
for subscription in subscriptions_to_invoice:
subscription.create_invoice()
subscription.next_invoice_date += relativedelta(month=1)
self.env.cr.commit()
# Use case 2
def _sync_partner(self):
while True:
partners = self.search([('dirty', '=', True)],
limit=100)
if not partners:
break
data = partners._prepare_data()
self._send_data(data)
partners.write({'dirty': False})
self.env.cr.commit()
Split in small batches and save at the end
of each batch.
➔ Only possible if already idempotent
+ Timeout will be resolved eventually
+ Reduce transaction time, lower
concurrent update risk
! Data consistency at commit
! Commit are annoying for automated
testing
! Run more often than you would to
avoid timeout

16. Robust
def _run_subscription_invoice(self):
subscriptions_to_invoice = self.search([
('stage_id.category', '=', 'progress'),
('next_invoice_date', '=', today),
])
for subscription in subscriptions_to_invoice:
try:
subscription.create_invoice()
subscription.next_invoice_date +=
relativedelta(month=1)
self.env.cr.commit()
except Exception:
self.env.cr.rollback()
def _sync_partner(self):
while True:
partners = self.search([('dirty', '=', True)],
limit=100)
if not partners:
break
try:
data = partners._prepare_data()
self._send_data(data)
partners.write({'dirty': False})
self.env.cr.commit()
except Exception:
self.env.cr.rollback()
Prevent an issue on a single record to lead
to a general failure.
➔ Crash != Timeout, there is no retry
➔ Keep data consistency: rollback
+ Process all correct records
! Record processing can fail silently
! Be able to filter those who failed

17. Complete
def _run_subscription_invoice(self):
subscriptions_to_invoice = self.search([
('stage_id.category', '=', 'progress'),
# Today or before
('next_invoice_date', '<=', today),
])
for subscription in subscriptions_to_invoice:
try:
subscription.create_invoice()
subscription.next_invoice_date +=
relativedelta(month=1)
self.env.cr.commit()
except Exception:
self.env.cr.rollback()
Don’t leave a record never processed.
➔ If a record was not processed,
make sure it will be included next
time
+ Process all correct records
! Keep idempotence
! May end up processing forever bad
records
! Too many bad records can lead to
infinite retry

18. Verbose
def _sync_partner(self):
while True:
partners = self.search([('dirty', '=', True)],
limit=100)
if not partners:
Break
try:
data = partners._prepare_data()
self._send_data(data)
partners.write({'dirty': False})
self.env.cr.commit()
_logger.debug('Batch of partner is synced')
except Exception:
self.env.cr.rollback()
_logger.exception('Something went wrong')
Log what is happening.
➔ When something went wrong
➔ When processing is long
+ Processing records with an issue does
not fail silently anymore
! Don’t bloat the logs

19. Testable
def _run_subscription_invoice(self, auto_commit=True):
subscriptions_to_invoice = self.search([
('stage_id.category', '=', 'progress'),
('next_invoice_date', '<=', today),
])
for subscription in subscriptions_to_invoice:
try:
subscription.create_invoice()
subscription.next_invoice_date +=
relativedelta(month=1)
if auto_commit:
self.env.cr.commit()
_logger.debug('Invoice created')
except Exception:
if auto_commit:
self.env.cr.rollback()
_logger.exception('Failure with subscription')
Make automated testing easy.
➔ Tests are rolled back
➔ Not possible with commit
➔ Make possible to disable commit
+ No excuse not to test anymore
! Default behavior: Commit

20. Not stubborn def _run_subscription_invoice(self):
subscriptions_to_invoice = self.search([
('stage_id.category', '=', 'progress'),
('next_invoice_date', '<=', today),
# Leave 10 tries, then drop it
('next_invoice_date', '>', 10_days_ago),
...
def _sync_partner(self):
# dirty is an int, will be set to 5
# Try until dirty reaches 1
# 0: ok
# 1: issue after max retry
for partner in self.search([('dirty', '>', 1)])
try:
data = partner._prepare_data()
self._send_data(data)
partner.write({'dirty': 0})
self.env.cr.commit()
except Exception:
self.env.cr.rollback()
partner.dirty -= 1
self.env.cr.commit()
Don’t retry bad records forever.
➔ Useful if a lot of bad records or bad
records that will never be fixed
➔ Implementation case by case
+ Avoid wasting time to process those
records
+ Resolved Time Out eventually
! Jeopardize completeness
! Easy way to find dropped records
! Analyse real usage to find the good
trade off

21. 4. Conclusion

22. Must Have
➔ Idempotent
➔ Incremental
➔ Robust
➔ Complete
Should have
➔ Verbose
➔ Testable
Could Have
➔ Not stubborn
Resilient Cron:
the recipe

23. Thank You