Spark Summit Europe 2017 Applying Multiple ML Pipelines to heterogenous data streams This talk explains how we adapted Spark mllib to deploy hundreds of ML pipelines in one streaming job to make real time predictions on heterogenous data streams.

1. Applying multiple
ML Pipelines
to heterogenous data streams
Gevorg Soghomonian, AI Research Engineer
Maciej Dabrowski, Chief Data Scientist
1

2. About Altocloud
2

3. • Understand customers better
• Improve customer experience
• Predict customer behaviour and engage
• Accelerate revenue conversion
• Reduce shopping cart abandons
Customer journey analytics

4. EVENT STREAMS EVENT PROCESSORS
Altocloud platform
BATCH
MODEL
LEARNING
ENRICHMENT PREDICTIONS
STORAGE
QUEUES
WEB EVENTS
CALL, IVR, MESSAGE EVENTS
ACTIONS
SEGMENTATION
Web
Hook
AGGREGATION
ACTION
CREATION
OUTCOME PROBABILITIES
REAL-TIME CUSTOMER JOURNEY
TICKET, LEAD,
GENERIC EVENTS
CONTEXT

5. Heterogenous data
5

6. Background
6

7. Spark Machine Learning framework
7

8. Spark Pipeline
8

9. Spark Transformers
High level interface for all operations on Datasets
Act as stages in Pipelines
Require various configuration: inputColumn, outputColumn, data, …
Dataset is the only input/output format
9

10. The challenge
10

11. Spark Pipeline deployment flow
11

12. Altocloud deployment flow
12

13. Altocloud deployment flow
13
How to operationalise
hundreds of models in one
Spark Streaming job?

14. Possible (deployment) solutions
Serialisation
✓ Ability to use any streaming technology
x Increased complexity
x Potentially increased deployment latency
Spark ML Pipelines
✓ Simplicity - one technology for training and evaluation
x Support only for the “one model scenario”
14

15. Heterogeneous datasets in Spark
Spark Pipelines can be applied only to Datasets
Spark Pipelines cannot be combined in a composite pipeline to be
applied on Pipeline-per-row basis
Questions:
How to apply different Pipeline(s) for each row in the Datasets?
How to identify which Pipelines should be applied to a row?
15

16. Applying hundreds of models to
heterogenous streams in
Spark Streaming
16

17. The foundation
Redefine Transformer API to allow compositional flexibility.
Adapt Spark Pipelines to redefined Transformers
17

18. Transformer API definition
18

19. One-to-One correspondence / adaptation
19

20. Adapt DataFrame Pipeline to Stream Pipeline
20

21. Adapt & combine
21

22. Composition example
22

23. Composition example
23

24. Apache Spark ML Pipeline flow revisited
24

25. Challenges
Re-implementation of transformer logic
Train-serving skew
Scaling beyond 1000s of models
25

26. Summary
26

27. Summary
Applying Pipelines to heterogenous data streams is hard.
Spark Pipelines cannot be combined and applied in a composite pipeline
on Pipeline-per-row basis
Our solution:
Extend Transformer API to apply different Pipeline(s) for each row
Extend Streams to include Pipeline composition
macdab@altocloud.com & gevorg@altocloud.com
27