How we applied continuous delivery to data science to create a high-performance & quickly evolving data product. Presented at Predictive Analytics World Business London 2016 by Arif Wider (ThoughtWorks) and Christian Deger (AutoScout24).

1. Predictive Analytics World London| Oct 12, 2016 | Arif Wider & Christian Deger
Predictive Analytics for Vehicle Price Prediction
Delivered Continuously at AutoScout24

2. Dr. Arif Wider
Senior Consultant/Developer
awider@thoughtworks.com
@arifwider

3. Christian Deger
Chief Architect
cdeger@autoscout24.com
@cdeger

4. PL
S
RUS
UA
RO
CZ
D
NL
B
F
A
HR
I
E
BG
TR
18countries
2.4m+cars & motos
10m+users per
month

5. The task: A consumer-facing data product
5Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

6. The task: A consumer-facing data product
6Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

7. The task: A consumer-facing data product
7Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

8. The prediction model: Random forest
8
Volkswagen GolfCar listings of
last two years
Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

9. How to turn an R-based prediction model
into a high-performance web application?
9
?
Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

10. How to turn an R-based prediction model
into a high-performance web application?
10Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

11. How to turn an R-based prediction model
into a high-performance web application?
11Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

12. How to turn an R-based prediction model
into a high-performance web application?
12
à Continuous Delivery!
Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

13. Application code in
one repository per
service.
CI
Deployment package
as artifact.
CD
Deliver package to
servers
Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger
Typical delivery pipeline

14. Continuous delivery pipelines
14
Prediction Model Pipeline
Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

15. Continuous delivery pipelines
15
Prediction Model Pipeline
Web Application Pipeline
Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

16. The price for CD: Extensive model validation
16Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

17. The price for CD: Extensive model validation
17Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

18. Lessons learned
18
§ Form cross-functional teams of
data scientists & software engineers!
§ Set up usage reporting early to improve your
data product in a data-driven way.
§ Generating gigabytes of Java code
is a challenge for the JVM
à Use the G1 garbage collector
à Do extensive warm-ups
àTurn off Tiered Compilation
Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

19. Conclusions
19
§ Continuous Delivery allows us to bring prediction
model changes live very quickly.
§ Only extensive automated end-to-end tests provide
confidence to deploy to production automatically.
§ Java code generation allows for very low response
times and excellent scalability for high loads but
requires plenty of memory.
Predictive Analytics World London 2016 Predictive Analytics Delivered Continuously – A. Wider & C. Deger

20. Thank you!
Questions?
– Arif Wider & Christian Deger –