A presentation of our next generation system architecture for data analytics. The Continuous Deep Analytics project aims to provide system support for end-to-end high-performance data stream analytics for ML and AI to facilitate critical decision making.

1. RISE Research Institutes of Sweden
knowledge
Computer Systems Laboratory
Paris Carbone - parisc@kth.se
PhD Candidate @ KTH
Committer @ Apache Foundation
CONTINUOUS
DEEP
ANALYTICS
RISE Open House 2018

2. !2
knowledge
Data
it is raining
it is cold
those who listen to A also like B
someone is crossing the street
credit fraud detected
users hate the website redesign
water/light sensor events
temperature sensor events
app-user likes
web user clicks
vehicle camera feed
user data and transactions
a storm is approachingclimate data and simulations
PROCESSING

3. !3
knowledge
Decision
Making
it is raining bring an umbrella
it is cold turn heating on
those who listen to A also like B recommend B to fans of A
someone is crossing the street stop the vehicle
credit fraud detected cancel transactions
users hate the website redesign switch to the old version
a storm is approaching alert boats!
REASONING

4. !4
complexity
> data dependencies
< response time
time criticality

5. !5
> data dependencies
it is raining
it is cold
those who
listen to A
will love B
someone is
crossing the street
credit fraud
people hate
the new menu
a storm approaches
< response time
complexity
time criticality

6. !6
CEP
bulk synchronous
iterative processing
deep neural
networks
graph analysis
derivative approximation
simulations
> data dependencies
< response time
complexity
time criticality
relational
algebra online ML

7. CEP
relational
algebra online ML
!7
bulk synchronous
iterative processing
deep neural
networks
graph analysis
derivative approximation
simulations
> data dependencies
< response time
complexity
time criticality
Tensor Programming
Platforms
(GPUs/TPUs)
Batch Processing
Platforms
(mem/CPU)
High Performance
Computing Platforms
(petaflops)
Data Stream/DBMS Platforms
(mem/CPU)

8. CEP
relational
algebra online ML
!7
bulk synchronous
iterative processing
deep neural
networks
graph analysis
derivative approximation
simulations
> data dependencies
< response time
complexity
time criticality
Tensor Programming
Platforms
(GPUs/TPUs)
Batch Processing
Platforms
(mem/CPU)
High Performance
Computing Platforms
(petaflops)
Data Stream/DBMS Platforms
(mem/CPU)
Continuous Deep Analytics

9. ▪ Modern Data Pipelines need to combine diverse workloads!
(ML Training & Serving, Relational Algebra, Streams, Tensors, Graphs)
!8
⋈
⋈
⋈
σθ
σθ
σθ
σθ
π
π
Relational Data Streams
Feature Learning
Tensor Programming Dynamic
Graphs

10. !9
Hardware Acceleration: Important but Not Enough
Pipeline (CPU)
Pipeline (GPU/TPU)
time until decision
critical decision
making

11. !10
Cross-Platform Computation is Inefficient
Stream
Tasks
Tensor
Tasks
Graph
Tasks
computationcomputation computation

12. !10
Cross-Platform Computation is Inefficient
Stream
Tasks
Tensor
Tasks
Graph
Tasks
computationcomputation computation
- No computation sharing optimisations

13. !10
Cross-Platform Computation is Inefficient
Stream
Tasks
Tensor
Tasks
Graph
Tasks
computationcomputation computation
- expensive data exchange through disk
- No computation sharing optimisations

14. !11
Computation Sharing
f f’…. ….Data knowledge

15. !11
Computation Sharing
f f’…. ….Data knowledge

16. !11
Computation Sharing
f f’
f + f’
loop fusion
vectorization
….
+less data passes
+no additional overhead
…. ….Data knowledge

17. !12
Critical Decision Making demands Efficiency
time until decision
Pipeline (CPU)
Pipeline (GPU/TPU)
critical decision
making

18. !12
Critical Decision Making demands Efficiency
Pipeline (CPU) - Optimised
Pipeline (GPU/TPU)
- Optimised
time until decision
Pipeline (CPU)
Pipeline (GPU/TPU)
critical decision
making

19. !13
The Problem
f f’
?
…. ….Data knowledge

20. !14

21. !15
Using an Intermediate Representation (IR)
f f’…. ….Data knowledge

22. !15
Using an Intermediate Representation (IR)
f f’…. ….Data knowledge
f+f’
IR IR
IR
f f’

23. !16
1) Weld IR (Stanford DAWN Project)
Related IR Projects
Matei Zaharia (Spark architect) et. al.
!16

24. !16
1) Weld IR (Stanford DAWN Project)
Related IR Projects
+ supports large number of existing libraries
- currently limited to short-lived local task execution
Matei Zaharia (Spark architect) et. al.
!16

25. !17
2) Tensor Comprehensions (Facebook)
Related IR Projects

26. !17
2) Tensor Comprehensions (Facebook)
Related IR Projects
+ JIT compiler, multi-GPU autotuner
- overspecialized to tensor operations in CUDA

27. !18
⋈
σθ
σθ
π
Dynamic
Graphs
Tensors
Relational
Streams
Data Pipeline
Our Goals: Model and Language Independence

28. !18
⋈
σθ
σθ
π
Dynamic
Graphs
Tensors
Relational
Streams
Data Pipeline
Our Goals: Model and Language Independence

29. !18
⋈
σθ
σθ
π
Dynamic
Graphs
Tensors
Relational
Streams
Data Pipeline
Our Goals: Model and Language Independence
…

30. !19
A Distributed Runtime for Heterogeneous Hardware
CDA Platform
Worker Worker Worker
LLVM code
Rust-Based

31. !20
Intermediate
Representation (IR)
Logically Optimised
IR
Compilation Steps

32. !21
Intermediate
Representation (IR)
Logically Optimised
IR
Physically Optimised
IR
Compilation Steps

33. !22
Physically Optimised
IR
Distributed JIT Compilation and Reconfiguration
Rust AST
LLVM

34. !23
The Current CDA Team (RISE SICS + KTH)
Computer
Systems
Machine
Learning
Lars
Kroll
Paris
Carbone
Christian
Schulte
Seif
Haridi
Theodore
Vasiloudis
Daniel
Gillblad
MSc Students
• Klas Segeljakt
• Oscar Bjuhr
• Johan Mickos

35. !24
Thank You!
Don’t miss our poster :)