A presentation delivered by Zalando engineers Joern Horstmann and Lukas Niemeier, September 2015.

1. Order Processing at Scale
Camunda Community Day 2015

2. 15 countries
3 fulfillment centers
16+ million active customers
2.2+ billion € revenue 2014
135+ million visits per month
9,000+ employees
ONE of EUROPE’S LARGEST ONLINE FASHION RETAILERS
Visit us: tech.zalando.com

4. Two Years Ago:
● Homegrown process framework
● Defined in XML
● Supported only minimal flow control
● Lots of special cases hidden in the code
● Asynchronous processing in batches

5. Sales Order Process (Synchronous)
● Begins after customer clicks “Buy Now”
● ~25 service tasks / ~30 gateways
○ Calculating accounting data for the ERP system
○ Validating coupons
○ Communicating with payment providers
○ Reserving stock

6. Sales Order Process (Async Part)
● Ongoing migration project
● ~10 service tasks / ~15 gateways
○ Sending confirmation mails
○ Fraud checks
○ Communicating with ERP system

7. Sharded Database
Distributing order and customer data across
multiple Postgres databases

8. Sharded Database
● Process engine data next to order data
● Allows joins between processes and orders
● Very useful during migration
○ Skip legacy process if there is an active process in
at least version x.y

9. Architecture
DB1
Shop
WS1 WS2 ... WSN
DB2 ... DBN
Job1 Job2 ... JobN

10. Separating WebService and Job Nodes
● Jobs do not influence synchronous process
● Can tune connection pools differently
● Leads to OptimisticLockingExceptions
○ Still works fine under load
○ Several improvements in 7.2 and 7.3

11. Deployment
● Rolling deployment of nodes
● Updated process model in DB while nodes
still run old code
● Code has to specify exact version to run
● Feature now included in release 7.3

12. Custom History Backend
● Not logging activity history to DB
● Improved performance due to fewer inserts
● Integrated into business event monitoring

13. Incident Management
$ ./cockpit-client.py
usage: cockpit-client.py [-h] (-s SHARD | -a)
[-e {live,integration,release-staging}]
[-u [USERNAME]] [-p [PASSWORD]]
[-i PROCESS_INSTANCE_ID] [-m MESSAGE]
[--from-timestamp FROM_TIMESTAMP]
[--to-timestamp TO_TIMESTAMP]
(-c | -C | -r | -l)
[-v]
$ ./cockpit-client.py -e live -a -m "could not send mail" -r
resolved incident for process instance 3d8121e1-4d7b-45df-8213-ad6e22484ee6

14. Implementation - Shard Aware
context().setup(order);
...
public class ShardAwareProcessEngine extends DelegatingProcessEngine {
private Map<String, ProcessEngine> processEngines;
@Override
protected ProcessEngine getProcessEngine() {
final Order order = context().getOrder();
return processEngines.get(order.getShardId());
}
}

15. Implementation - Custom Scopes
@Component
@FlowScoped
public class SendOrderConfirmationMail implements JavaDelegate
@Autowired
private Order currentOrder;
@Autowired
private MailService mailService;
@Override
public void execute(final DelegateExecution execution) { }
}

16. ● Tech Blog: tech.zalando.com
● GitHub: github.com/zalando
● Twitter: @ZalandoTech
● Instagram: zalandotech
CHECK US OUT!
Jobs: http://tech.zalando.com/jobs