These are the slides for the lecture I gave at the EBSIS Summer School about data streaming and its challenges and trade-offs for data analysis in Fog architectures.

1. The data streaming paradigm
and its use in Fog architectures
Vincenzo Gulisano, Ph.D.
EBSIS Summer School 2016
1

2. Agenda
• Introduction: Fog architectures
• Data streaming basics
• Trade-offs and challenges in Fog streaming analysis
• Conclusions
• Bibliography
2The data streaming paradigm and its use in Fog architectures

3. Agenda
• Introduction: Fog architectures
• Data streaming basics
• Trade-offs and challenges in Fog streaming analysis
• Conclusions
• Bibliography
3The data streaming paradigm and its use in Fog architectures

4. Introduction: Fog architectures
Advanced Metering Infrastructures (AMIs)
[1,2,3,4]
Vehicular Networks (VNs)
[5,6]
4The data streaming paradigm and its use in Fog architectures

5. AMIs VNs
• demand-response
• scheduling [7]
• micro-grids
• detection of medium size blackouts [8]
• detection of non technical losses
• ...
5
• autonomous driving
• platooning
• accident detection [9]
• variable tolls [9]
• congestion monitoring [10]
• ...
Access to their data à Increased awareness, security, power-efficiency, ...
The data streaming paradigm and its use in Fog architectures

6. (some) fundamental questions
• where do we process data?
• how do we process data?
• how do we enforce security and privacy?
6The data streaming paradigm and its use in Fog architectures

7. the wrong incomplete answer we run the analysis in the cloud!
7
1. does the infrastructure allow for billions of
readings per day to be transferred continuously?
2. the latency incurred while transferring data, does
that undermine the utility of the analysis?
3. is it secure to concentrate all the data in a single
place? [11]
4. is it smart to give away fine-grained data? [12]
The data streaming paradigm and its use in Fog architectures

8. A small example of what fine-grained data can reveal...
8
source: Private Memoirs of a Smart Meter [12]
The data streaming paradigm and its use in Fog architectures

9. a better answer we leverage the entire infrastructure!
9
Traditional analysis techniques cannot address
all the challenges in these setups [13,14]
The data streaming paradigm and its use in Fog architectures

10. Fog architectures
10
source: Fog Computing and Its Role in the Internet of Things [15]
Characteristics [15]:
1. edge location / location
awareness / low latency
2. geographical distribution
3. large-scale
4. support for mobility
5. real-time interactions
6. predominance of wireless
7. heterogeneous
8. interoperability / federation
9. interaction with the cloud
The data streaming paradigm and its use in Fog architectures

11. Fog architectures, applications trade-offs
and challenges
11
Sensor/Edge Devices Fog Devices Cloud Devices
Cost + ++ +++1
Computational power + ++ +++
Inter-node heterogeneity +++ ++ +
Intra-node heterogeneity + + +++
Size +++ ++ +
Communication wireless wireless/wired wired
Volume of data millions of small streams2 thousands of medium aggregated streams many large streams
Evolution over time +++ ++ +3
1. depends on who owns the physical hardware
2. with exceptions like e.g. Lidar sensors [16,17]
3. Virtual environments can also evolve ”behind the scenes”
The data streaming paradigm and its use in Fog architectures

12. Agenda
• Introduction: Fog architectures
• Data streaming basics
• Trade-offs and challenges in Fog streaming analysis
• Conclusions
• Bibliography
12The data streaming paradigm and its use in Fog architectures

13. Motivation
Applications in sensor networks, cyber-physical
systems:
• large and fluctuating volumes of data generated
continuously
demand for:
• Continuous processing of data streams
• In a real-time fashion
Store-then-process is not feasible!!!
13The data streaming paradigm and its use in Fog architectures

14. Main Memory
Motivation
DBMS vs. DSMS
Disk
1 Data
Query Processing
3 Query
results
2 Query
Main Memory
Query Processing
Continuous
Query
Data
Query
results
14The data streaming paradigm and its use in Fog architectures

15. Before we start... about data streaming and Stream Processing Engines (SPEs)
15
An incomplete, non-sorted list of SPEs (cf. related work in [18]):
time
Borealis
The Aurora Project
STanfordstREamdatAManager
NiagaraCQ
COUGAR
StreamCloud
Covering all of them / discussing which use cases are best for each one out of scope...
will focus on the ones I work with and on example from vehicular networks
The data streaming paradigm and its use in Fog architectures

16. data stream: unbounded sequence of tuples sharing the same schema
16
Example: vehicles’ speed reports
time
Field Field
vehicle id text
time (secs) text
speed (Km/h) double
X coordinate double
Y coordinate double
A 8:00 55.5 X1 Y1
Let’s assume each source
(e.g., vehicle) produces
and delivers a timestamp
sorted stream
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
The data streaming paradigm and its use in Fog architectures

17. continuous query (or simply query): Directed Acyclic Graph (DAG) of
streams and operators
17
OP
OP
OP
OP OP
OP OP
source op
(1+ out streams)
sink op
(1+ in streams)
stream
op
(1+ in, 1+ out streams)
The data streaming paradigm and its use in Fog architectures

18. data streaming operators
Two main types:
• Stateless operators
• do not maintain any state
• one-by-one processing
• if they maintain some state, such state does not evolve depending
on the tuples being processed
• Stateful operators
• maintain a state that evolves depending on the tuples being
processed
• produce output tuples that depend on multiple input tuples
18
OP
OP
The data streaming paradigm and its use in Fog architectures

19. stateless operators
19
Filter
...
Map
Union
...
Filter / route tuples based on one (or more) conditions
Transform each tuple
Merge multiple streams (with the same schema) into one
The data streaming paradigm and its use in Fog architectures

20. stateless operators
20
Filter
...
Map
Union
...
source: http://storm.apache.org/releases/2.0.0-SNAPSHOT/Trident-tutorial.html
The data streaming paradigm and its use in Fog architectures

21. stateless operators
21
Filter
...
Map
Union
...
source: https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/index.html
The data streaming paradigm and its use in Fog architectures

22. stateless operators
22
Filter
...
Map
Union
...
source: http://spark.apache.org/docs/latest/streaming-programming-guide.html#transformations-on-dstreams
The data streaming paradigm and its use in Fog architectures

23. stateful operators
23
Aggregate information from multiple tuples
(e.g., max, min, sum, ...)
Join tuples coming from 2 streams given a certain predicate
Aggregate
Join
The data streaming paradigm and its use in Fog architectures

24. stateful operators
24
source: http://storm.apache.org/releases/2.0.0-SNAPSHOT/Trident-tutorial.html
source: http://spark.apache.org/docs/latest/streaming-programming-
guide.html#transformations-on-dstreams
source: http://spark.apache.org/docs/latest/streaming-programming-
guide.html#transformations-on-dstreams
The data streaming paradigm and its use in Fog architectures

25. Wait a moment!
if streams are unbounded, how can we aggregate or join?
25The data streaming paradigm and its use in Fog architectures

26. windows and stateful analysis [18]
Stateful operations are done over windows:
• Time-based (e.g., tuples in the last 10 minutes)
• Tuple-based (e.g., given the last 50 tuples)
26
time
[8:00,9:00)
[8:20,9:20)
[8:40,9:40)
Usually applications rely on time-based sliding windows
The data streaming paradigm and its use in Fog architectures

27. time-based sliding window aggregation (count)
27
Counter: 4
time
[8:00,9:00)
8:05 8:15 8:22 8:45 9:05
Output: 4
Counter: 1
Counter: 2
Counter: 3
Counter: 3
time
8:05 8:15 8:22 8:45 9:05
[8:20,9:20)
we assumed each source
produces and delivers a
timestamp sorted stream!
What happens if this is not
the case?
The data streaming paradigm and its use in Fog architectures

28. time-based sliding window joining
28
t1
t2
t3
t4
t1
t2
t3
t4
R S
Sliding
window Window
size WS
WSWR
Predicate P
The data streaming paradigm and its use in Fog architectures

29. 29
windows and stateful analysis
The data streaming paradigm and its use in Fog architectures

30. 30
basic operators and user-defined operators
Besides a set of basic operators, SPEs usually allow the user to define
ad-hoc operators (e.g., when existing aggregation are not enough)
The data streaming paradigm and its use in Fog architectures

31. sample query
For each vehicle, raise an alert if the speed of the latest report is more
than 2 times higher than its average speed in the last 30 days.
31
time
A 8:00 55.5 X1 Y1 A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
The data streaming paradigm and its use in Fog architectures

32. 32
Remove
unused fields
Map
Field
vehicle id
time (secs)
speed (Km/h)
X coordinate
Y coordinate
Field
vehicle id
time (secs)
speed (Km/h)
Compute average
speed for each
vehicle during the
last 30 days
Aggregate
Field
vehicle id
time (secs)
avg speed (Km/h)
Join
Check
condition
Filter
Field
vehicle id
time (secs)
speed (Km/h)
Join on
vehicle id
Field
vehicle id
time (secs)
avg speed (Km/h)
speed (Km/h)
sample query
The data streaming paradigm and its use in Fog architectures

33. 33
M A J F
sample query
Notice:
• the same semantics can be defined in several ways (using different
operators and composing them in different ways)
• Using many basic building blocks can ease the task of distributing
and parallelizing the analysis (more in the following...)
The data streaming paradigm and its use in Fog architectures

34. Agenda
• Introduction: Fog architectures
• Data streaming basics
• Trade-offs and challenges in Fog streaming analysis
• Conclusions
• Bibliography
34The data streaming paradigm and its use in Fog architectures

35. 1. Distributed deployment
2. Parallel deployment
3. Ordering and determinism
4. Shared-nothing vs shared-memory parallelism
5. Load balancing
6. Elasticity
7. Fault tolerance
8. Data sharing (differential privacy)
35
8 aspects / use cases to discuss challenges (and opportunities)
The data streaming paradigm and its use in Fog architectures

36. Before we start...
36
Following examples are from vehicular networks
Road-side unit
RSU
Vehicle
Server
The data streaming paradigm and its use in Fog architectures

37. 1 - Distributed deployment – where to place a given operator? [19]
37
M
1. What hardware is available at each possible
deployment location?
2. What data is available at each possible
deployment location, depending on
1. the infrastructure?
2. its security regulations?
3. its privacy regulations?
?
The data streaming paradigm and its use in Fog architectures

38. 1 - Distributed deployment – where to place a given operator?
38
M
1. Low latency (if all the required data is locally
accessible)
2. Access to all local data (parts of which might
not be reported to RSUs / Servers) [4]
3. Embedded devices / limited computational
power [20]
The data streaming paradigm and its use in Fog architectures

39. 1 - Distributed deployment – where to place a given operator?
39
M
1. Access to multiple vehicles / high mobility
2. Access to smaller sets of data / partial view of
the data [22]
3. Embedded devices / higher computational
power
4. Wireless / wired communication
The data streaming paradigm and its use in Fog architectures

40. 1 - Distributed deployment – where to place a given operator?
40
1. Global view of the data (when possible [22])
2. Access to aggregated data
3. High computational power
The data streaming paradigm and its use in Fog architectures
M

41. 2 - Parallel deployment – how do we parallelize the analysis?
41
If a parallel operator (e.g., M) feeds multiple downstream
parallel operators (e.g., A), how do we route tuples? [23,24]
M
M A
A
The data streaming paradigm and its use in Fog architectures

42. 2 - Parallel deployment – how do we parallelize the analysis?
42
M
M A
A
A 8:00 55.5 X1 Y1
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
A 8:00 55.5 X1 Y1
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
When computing e.g.
per-vehicle statistics,
we need to make sure
that all the tuples
referring to the same
vehicles are sent to
the same aggregate
instance
The data streaming paradigm and its use in Fog architectures

43. • Depending on the stateful operator semantic:
• Partition input stream into buckets
• Each bucket is processed by 1 node
• # buckets >> # nodes
43
2 - Parallel deployment – how do we parallelize the analysis?
The data streaming paradigm and its use in Fog architectures

44. • Depending on the stateful operator semantic:
• Partition input stream into buckets
• Each bucket is processed by 1 node
• # buckets >> # nodes
Keys
domain
Agg1 Agg2 Agg3
A
D
E
B
C F
44
2 - Parallel deployment – how do we parallelize the analysis?
The data streaming paradigm and its use in Fog architectures

45. • Depending on the stateful operator semantic:
• Partition input stream into buckets
• Each bucket is processed by 1 node
• # buckets >> # nodes
Keys
domain
Agg1 Agg2 Agg3
A
D
E
B
C F
45
2 - Parallel deployment – how do we parallelize the analysis?
The data streaming paradigm and its use in Fog architectures

46. 2 - Parallel deployment – how do we parallelize the analysis?
46
How can we parallelize the analysis of e.g. an aggregate when
the operations are not performed for disjoint partitions of a
stream (e.g., for all vehicles rather than for each vehicle?)
M
M A
A
The data streaming paradigm and its use in Fog architectures

47. 47
2 - Parallel deployment – how do we parallelize the analysis?
The data streaming paradigm and its use in Fog architectures

48. 48
M
M A
A
A 8:00 55.5 X1 Y1
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
A 8:00 55.5 X1 Y1
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
What if tuple with
timestamp 8:00
arrives after tuple
with timestamp 8:07?
3 – Ordering and determinism
The data streaming paradigm and its use in Fog architectures

49. 49
3 – Ordering and determinism
if tuples that aggregated together are arbitrarily interleaved,
how to we provide correct results?
time
[8:00,9:00)
8:05 8:15 8:22 8:45 9:05
Output: 4
time
[8:00,9:00)
8:05 8:15 8:22 8:45 9:058:55
The data streaming paradigm and its use in Fog architectures

50. 50
3 – Ordering and determinism – merge-sorting [25] & buffering [26,27]
addTuple(tuple,sourceID)
allows a tuple from sourceID to be merged by ScaleGate in the
resulting timestamp-sorted stream of ready tuples.
getNextReadyTuple(readerID)
provides to readerID the next earliest ready tuple that has not been
yet consumed by the former.
https://github.com/dcs-chalmers/ScaleGate_Java
The data streaming paradigm and its use in Fog architectures

51. 4 – shared-nothing vs. shared-memory parallelism [28]
51
M
M
...
A
A
J
...
How to take advantage of multi-core
architectures?
How to boost inter-node parallelism
and intra-node parallelism?
The data streaming paradigm and its use in Fog architectures

52. Shared-nothing parallel stream join
Prod
R
Prod
S
PU1
PU2
PUN…
Cons
Add tuple to PUi S
Add tuple to PUi R
Consume results
Pick the next ready tuple t:
1. compare t with all tuples in opposite window given P
2. add t to its window
3. remove stale tuples from t’s window
Chose a PU
Chose a PU
Take the next
ready output tuple
Scalability
High
throughput
Low latency
Disjoint
parallelism
Skew
resilience
Determinism
52
Merge
4 – shared-nothing vs. shared-memory parallelism
The data streaming paradigm and its use in Fog architectures

53. Prod
R
Prod
S
PU1
PU2
PUN
…
53
enqueue()
dequeue()
ConsMerge
Shared-nothing parallel stream join
4 – shared-nothing vs. shared-memory parallelism
The data streaming paradigm and its use in Fog architectures

54. Prod
R
Prod
S
PU1
PU2
PUN
…
Cons
Add tuple SGin
Add tuple SGin
Get next ready
output tuple
from SGout
Get next ready input tuple from SGin
1. compare t with all tuples in opposite window given P
2. add t to its window in a round-robin fashion
3. remove stale tuples from t’s window
54
SGin SGout
Steps for PU
Shared-memory parallel stream join
4 – shared-nothing vs. shared-memory parallelism
The data streaming paradigm and its use in Fog architectures

55. 55
t1
t2
R S
WR
t3
t4
R S
t4
t1
WR
R S
t4
t2
WR
R S
t4
WR
t3
Sequential stream join:
ScaleJoin with 3 PUs:
Shared-memory parallel stream join
4 – shared-nothing vs. shared-memory parallelism
The data streaming paradigm and its use in Fog architectures

56. Prod
R
Prod
S
PU1
PU2
PUN
…
Cons
Add tuple SGin
Add tuple SGin
Get next ready
output tuple
from SGout
56
SGin SGout
Scalability
High
throughput
Low latency
Disjoint
parallelism
Skew
resilience
Determinism
Prod
S
Prod
S
Prod
R
Get next ready input tuple from SGin
1. compare t with all tuples in opposite window given P
2. add t to its window in a round robin fashion
3. remove stale tuples from t’s window
Steps for PUi
Shared-memory parallel stream join
4 – shared-nothing vs. shared-memory parallelism
The data streaming paradigm and its use in Fog architectures

57. 5 – load balancing & state transfer [23,29]
57
If we shift the processing of a certain subset of
tuples from node A to node B, how do transfer
its previous state?
M
A
M
A
The data streaming paradigm and its use in Fog architectures

58. State transfer challenging for stateful operators
A B
time
58
5 – load balancing & state transfer
The data streaming paradigm and its use in Fog architectures

59. Window Recreation Protocol [23]
A B
time
A
A
B
Send to A
Send to B
59
5 – load balancing & state transfer
+ No communication
between nodes
- Completion time
proportional to window
size
The data streaming paradigm and its use in Fog architectures

60. State Recreation Protocol
A B
time
B
B
B
Copy to B
60
Tuples referring
to caller A
5 – load balancing & state transfer
+ Minimizes completion
time
- Communication
between nodes
The data streaming paradigm and its use in Fog architectures

61. 6 – elasticity [23,30]
61
How / when to provision or
decommission new resources depending
on the analysis’ costs fluctuations?
J
The data streaming paradigm and its use in Fog architectures
J J
J

62. Target utilization threshold
Upper utilization threshold
Lower utilization threshold
0%
100%
CPU consumption
Memory
Latency
...
………
OP OP OPOP OP
Elasticity and load balancing actions on per-subcluster basis
62
6 – elasticity [23]
The data streaming paradigm and its use in Fog architectures

63. 63
6 – elasticity
• How many resources should be provisioned /
decommissioned? [23]
• How to mix load balancing and provisioning /
decommissioning? [23]
• How to chose which resources to provision /
decommission? [30]
• How to predict when more / less resources will
be needed? [30]
The data streaming paradigm and its use in Fog architectures

64. Load-aware provisioning
1 2 4 7 11 2617
64
6 – elasticity [23]
The data streaming paradigm and its use in Fog architectures

65. 7 – fault tolerance [related work in 18, 31, 32]
65
How to replace a failed node minimizing
recovery time (making it transparent to
the end user)?
The data streaming paradigm and its use in Fog architectures
J
J
J
J
J
J

66. Active standby
Primary
Replica
AM
A
66
Cost:
Recovery
Time:
7 – fault tolerance
The data streaming paradigm and its use in Fog architectures

67. Passive standby
Primary
Replica
AM
A
Periodic
checkpoints
67
Cost:
Recovery
Time:
7 – fault tolerance
The data streaming paradigm and its use in Fog architectures

68. Passive standby
Primary
AM
Periodic
checkpoints
68
Cost:
Recovery
Time:
Disk
7 – fault tolerance
The data streaming paradigm and its use in Fog architectures

69. Upstream Backup
AM
Buffer
69
Cost:
Recovery
Time:
7 – fault tolerance
The data streaming paradigm and its use in Fog architectures

70. Fault tolerance and elasticity can also be integrated
together in solution that externalize the management
of state from the SPE [32]
70
7 – fault tolerance
The data streaming paradigm and its use in Fog architectures

71. 8 – data sharing (differential privacy) [2,33,34,35]
71
How to prevent privacy leaks?
Suppose we are interested in publishing vehicles’
average speed over a window of one hour...
We could aggregate by RSU!
The data streaming paradigm and its use in Fog architectures

72. 8 – data sharing (differential privacy)
72
Wait a moment!
what if a single vehicle is
connected to a certain RSU?
Whether a certain mechanism preserves or not the privacy
of the underlying data depends on the knowledge of the
adversary
Differential privacy assumes the worst case scenario!
The data streaming paradigm and its use in Fog architectures

73. 73
8 – data sharing (differential privacy)
time
[8:00,9:00)
22.3 Km/h 10 Km/h ? 15Km/h
avg speed: 30 Km/h
without differential privacy, if knows the speed of , and
and the avg speed, then it can find out the speed of
The data streaming paradigm and its use in Fog architectures

74. 74
8 – data sharing (differential privacy)
time
[8:00,9:00)
22.3 Km/h 10 Km/h ? 15Km/h
avg speed: 30 Km/h
with differential privacy, if knows the speed of , and , it does
gain extra knowledge from the avg speed in order to find the speed of
+ noise (calibrated from probabilistic distribution) so
that whether participates or not, the results
will be nearly the same
The data streaming paradigm and its use in Fog architectures

75. 75
8 – data sharing (differential privacy)
The data streaming paradigm and its use in Fog architectures

76. Agenda
• Introduction: Fog architectures
• Data streaming basics
• Trade-offs and challenges in Fog streaming analysis
• Conclusions
• Bibliography
76The data streaming paradigm and its use in Fog architectures

77. Introduction: Fog architectures
77
• where do we process data?
• how do we process data?
• how do we enforce security and privacy?
... cannot really answer to these questions
independently ...
The data streaming paradigm and its use in Fog architectures

78. Fog architectures, applications trade-offs
and challenges
78
Sensor/Edge Devices Fog Devices Cloud Devices
Cost + ++ +++
Computational power + ++ +++
Inter-node heterogeneity +++ ++ +
Intra-node heterogeneity + + +++
Size +++ ++ +
Communication wireless wireless/wired wired
Volume of data millions of small streams thousands of medium aggregated streams many large streams
Evolution over time +++ ++ +
The data streaming paradigm and its use in Fog architectures

79. Agenda
• Introduction: Fog architectures
• Data streaming basics
• Trade-offs and challenges in Fog streaming analysis
• Conclusions
• Bibliography
79The data streaming paradigm and its use in Fog architectures

80. Bibliography
1. Zhou, Jiazhen, Rose Qingyang Hu, and Yi Qian. "Scalable distributed communication architectures to support advanced
metering infrastructure in smart grid." IEEE Transactions on Parallel and Distributed Systems 23.9 (2012): 1632-1642.
2. Gulisano, Vincenzo, et al. "BES: Differentially Private and Distributed Event Aggregation in Advanced Metering Infrastructures."
Proceedings of the 2nd ACM International Workshop on Cyber-Physical System Security. ACM, 2016.
3. Gulisano, Vincenzo, Magnus Almgren, and Marina Papatriantafilou. "Online and scalable data validation in advanced metering
infrastructures." IEEE PES Innovative Smart Grid Technologies, Europe. IEEE, 2014.
4. Gulisano, Vincenzo, Magnus Almgren, and Marina Papatriantafilou. "METIS: a two-tier intrusion detection system for advanced
metering infrastructures." International Conference on Security and Privacy in Communication Systems. Springer International
Publishing, 2014.
5. Yousefi, Saleh, Mahmoud Siadat Mousavi, and Mahmood Fathy. "Vehicular ad hoc networks (VANETs): challenges and
perspectives." 2006 6th International Conference on ITS Telecommunications. IEEE, 2006.
6. El Zarki, Magda, et al. "Security issues in a future vehicular network." European Wireless. Vol. 2. 2002.
7. Georgiadis, Giorgos, and Marina Papatriantafilou. "Dealing with storage without forecasts in smart grids: Problem
transformation and online scheduling algorithm." Proceedings of the 29th Annual ACM Symposium on Applied Computing.
ACM, 2014.
8. Fu, Zhang, et al. "Online temporal-spatial analysis for detection of critical events in Cyber-Physical Systems." Big Data (Big
Data), 2014 IEEE International Conference on. IEEE, 2014.
The data streaming paradigm and its use in Fog architectures 80

81. Bibliography
9. Arasu, Arvind, et al. "Linear road: a stream data management benchmark." Proceedings of the Thirtieth
international conference on Very large data bases-Volume 30. VLDB Endowment, 2004.
10. Lv, Yisheng, et al. "Traffic flow prediction with big data: a deep learning approach." IEEE Transactions on
Intelligent Transportation Systems 16.2 (2015): 865-873.
11. Grochocki, David, et al. "AMI threats, intrusion detection requirements and deployment
recommendations." Smart Grid Communications (SmartGridComm), 2012 IEEE Third International
Conference on. IEEE, 2012.
12. Molina-Markham, Andrés, et al. "Private memoirs of a smart meter." Proceedings of the 2nd ACM
workshop on embedded sensing systems for energy-efficiency in building. ACM, 2010.
13. Gulisano, Vincenzo, et al. "Streamcloud: A large scale data streaming system." Distributed Computing
Systems (ICDCS), 2010 IEEE 30th International Conference on. IEEE, 2010.
14. Stonebraker, Michael, Uǧur Çetintemel, and Stan Zdonik. "The 8 requirements of real-time stream
processing." ACM SIGMOD Record 34.4 (2005): 42-47.
15. Bonomi, Flavio, et al. "Fog computing and its role in the internet of things." Proceedings of the first
edition of the MCC workshop on Mobile cloud computing. ACM, 2012.
16. Himmelsbach, Michael, et al. "LIDAR-based 3D object perception." Proceedings of 1st international
workshop on cognition for technical systems. Vol. 1. 2008.
The data streaming paradigm and its use in Fog architectures 81

82. Bibliography
17. Geiger, Andreas, et al. "Vision meets robotics: The KITTI dataset." The International Journal of Robotics Research (2013):
0278364913491297.
18. Gulisano, Vincenzo Massimiliano. StreamCloud: An Elastic Parallel-Distributed Stream Processing Engine. Diss. Informatica,
2012.
19. Cardellini, Valeria, et al. "Optimal operator placement for distributed stream processing applications." Proceedings of the 10th
ACM International Conference on Distributed and Event-based Systems. ACM, 2016.
20. Costache, Stefania, et al. "Understanding the Data-Processing Challenges in Intelligent Vehicular Systems." Proceedings of the
2016 IEEE Intelligent Vehicles Symposium (IV16).
21. Cormode, Graham. "The continuous distributed monitoring model." ACM SIGMOD Record 42.1 (2013): 5-14.
22. Giatrakos, Nikos, AntoniosDeligiannakis, and Minos Garofalakis. "Scalable Approximate Query Tracking over Highly Distributed
Data Streams." Proceedings of the 2016 International Conference on Management of Data. ACM, 2016.
23. Gulisano, Vincenzo, et al. "Streamcloud: An elastic and scalable data streaming system." IEEE Transactions on Parallel and
Distributed Systems 23.12 (2012): 2351-2365.
24. Shah, Mehul A., et al. "Flux: An adaptive partitioning operator for continuous query systems." Data Engineering, 2003.
Proceedings. 19th International Conference on. IEEE, 2003.
The data streaming paradigm and its use in Fog architectures 82

83. Bibliography
25. Cederman, Daniel, et al. "Brief announcement: concurrent data structures for efficient streaming aggregation." Proceedings of
the 26th ACM symposium on Parallelism in algorithms and architectures. ACM, 2014.
26. Ji, Yuanzhen, et al. "Quality-driven processing of sliding window aggregates over out-of-order data streams." Proceedings of the
9th ACM International Conference on Distributed Event-Based Systems. ACM, 2015.
27. Ji, Yuanzhen, et al. "Quality-driven disorder handling for concurrent windowed stream queries with shared operators."
Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems. ACM, 2016.
28. Gulisano, Vincenzo, et al. "Scalejoin: A deterministic, disjoint-parallel and skew-resilient stream join." Big Data (Big Data), 2015
IEEE International Conference on. IEEE, 2015.
29. Ottenwälder, Beate, et al. "MigCEP: operator migration for mobility driven distributed complex event processing." Proceedings
of the 7th ACM international conference on Distributed event-based systems. ACM, 2013.
30. De Matteis, Tiziano, and Gabriele Mencagli. "Keep calm and react with foresight: strategies for low-latency and energy-efficient
elastic data stream processing." Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel
Programming. ACM, 2016.
31. Balazinska, Magdalena, et al. "Fault-tolerance in the Borealis distributed stream processing system." ACM Transactions on
Database Systems (TODS) 33.1 (2008): 3.
32. Castro Fernandez, Raul, et al. "Integrating scale out and fault tolerance in stream processing using operator state
management." Proceedings of the 2013 ACM SIGMOD international conference on Management of data. ACM, 2013.
The data streaming paradigm and its use in Fog architectures 83

84. Bibliography
33. Dwork, Cynthia. "Differential privacy: A survey of results." International Conference on Theory and Applications of
Models of Computation. Springer Berlin Heidelberg, 2008.
34. Dwork, Cynthia, et al. "Differential privacy under continual observation." Proceedings of the forty-second ACM
symposium on Theory of computing. ACM, 2010.
35. Kargl, Frank, Arik Friedman, and Roksana Boreli. "Differential privacy in intelligent transportation systems." Proceedings
of the sixth ACM conference on Security and privacy in wireless and mobile networks. ACM, 2013.
The data streaming paradigm and its use in Fog architectures 84

85. Thank you! Questions?
85The data streaming paradigm and its use in Fog architectures