Today, myriads of sensors are surrounding us. Their usage ranges from environmental monitoring (e.g., weather and air quality), over sensor-equipped smart buildings, to the quantified self and other human observing applications. The data streams produced by such sensors often update with high frequencies, resulting in large data volumes. Being able to analyze those real-time sensor data streams requires efficient visualization techniques. In our work, we explore how 3D visualizations can be used to extend the available information space. More specifically, we present an approach for processing real-time sensor data streams to enable scalable Web-based 3D visualizations. Based on an event-driven architecture, our key contribution is the presentation of three processing patterns to optimize transmission of sensor data streams to 3D Web clients.

1. ProcessingReal-time Sensor Data Streams for3D Web Visualization
Arne Bröring*, David Vial*, Thorsten Reitz**
* Esri Suisse
** Esri R&D Center Zurich
5thIWGS Workshop @ ACM Sigspatial
4thNovember 2014, Dallas, USA

2. © 2014 Esri Suisse
Wearemore& moresurroundedbysensors…
http://www.aaybee.com.au
http://mmminimal.com/nest-protect/

3. Gainingknowledgefrombigsensordata
Data
Information
Knowledge
Wisdom
© 2014 Esri Suisse
Web-based
3D Visual Analyticsfor
real-time sensordata
>3D engines(e.g. WebGL) renderframesusingstricttime budget.
3D engineshaveperformance& scalabilityissues, when:
directlypushinghigh frequency& irregularlydata
•Challenge:
>High dataratesandvolumes.
Aim:

4. © 2014 Esri Suisse
•Goal:
>processsensor data streams to enable
efficient data transmission to 3D visualization clients.
Gainingknowledgefrombigsensordata
Aim:

5. © 2014 Esri Suisse
•Design of:
1.event-drivenarchitecturetoprocesssensordatastreams
2.processingpatternsforefficienttransmissionofsensorstreamdatato3D clients
Approach

6. © 2014 Esri Suisse
Architecture
GeospatialData Server
Sensor Data Broker
3D VisualizationClient
Sensor
Feature Push
Service
3D Scene Download
Service
Sensor Data Push Service
Publishesmessagesto
Pulls3D scene
Pushesfeaturesto
Registers at
Forwards sensordatato

7. Pushesi3sfeaturesto
Pullsi3sscene
IBM Intelligent OperationsCenter
ArcGISServer
3D Client (e.g. Web Scene Viewer)
© 2014 Esri Suisse
ArchitectureImplementation
GeoEventProcessor
Scene Service
MQTT Broker
topic
Sensor
Registers at
Publishesmessagesto
Forwards sensordatato

8. © 2014 Esri Suisse
•Stands for: Indexed3D Scene
•Newly developed by Esri
•Format to stream 3D GIS data to mobile, web and desktop clients
•JSON encoded
•Default format of the ArcGIS Scene Service
•Clients:
>ArcGISWeb Scene Viewer
>ArcGISMobile Runtime
>ArcGISPro
i3s Format

9. © 2014 Esri Suisse
•Stands for: Message Queue TelemetryTransport
•Messaging protocol to realize pub/sub on top of the TCP/IP
•Light-weight; for limited processing power and/or network bandwidth
•Pub/sub requires message broker
•Broker distributes messages to clients based on topics
MQTT

10. © 2014 Esri Suisse
ArchitectureImplementation Details
MQTT InboundTransport
Text Adapter
GeoEventService
GeoEventProcessor
MQTT-Input
Connector
WebSocketOutbound Transport
i3s Outbound Adapter
i3s-Output
Connector
MQTT
i3s
IBM Intelligent OperationsCenter
ArcGISServer
3D Client (e.g. Web Scene Viewer)
GeoEvent
Processor
Scene
Service
MQTT Broker
Sensor

11. Patterns forSensor Data Stream Processing

12. © 2014 Esri Suisse
Sensors & Features
f1: [8:03:11; 22,34; 52N, 7W]
f2: [8:04:45; 18,43; 53N, 8W]
f3: [8:05:06; 19,63; 54N, 9W]
f2: [8:06:47; 19,12; 53N, 8W]
…
•High frequencyin data
•Irregulardatasending
•Large / smallfeatures
(varietyin characteristics)

13. © 2014 Esri Suisse

•Irregulardatatransmissionfromservertoclient
•Inefficient, e.g., whenlarge featuresaremostlyconstant, but onecharacteristicchanges
DirectData Transmission
f1
tstart
f1'
f2
f1''
f2'
f3
f3'
MQTT
i3s
tstart
f1
f1'
f2
f1''
f2'
f3
f3'

14. 
•Periodicaldatatransmissionfromservertoclient
© 2014 Esri Suisse
Pattern 1: Aggregation over Time Interval
tpX
f1
f1'
f2
f1''
f2'
f3
MQTT
i3s
tstart+tpx
f1
tstart
f1'
f2
f1''
f2'
f3
f3'

15. 
•Efficient, e.g., when very high numbers of messages / features are coming in to the server
Good if client only needs latest representation
tstart+ tpx
© 2014 Esri Suisse
Pattern 2: Sending Latest Feature Only
f1''
f2'
f3'
MQTT
i3s
tpX
f1
tstart
f1'
f2
f1''
f2'
f3
f3'

16. 
•Efficient when features have a lot of changes on single characteristics, while others remain constant.
Good if client only needs the differences
© 2014 Esri Suisse
Pattern 3: Sending Feature Differences Only
f1(A, B, C)
t1
f1(A’, B’, C)
t2
f1(A’, B’’, C)
t3
f1(A, B, C)
t1+x
f1(A’, B’ )
t2+y
f1( B’’ )
t3+z
MQTT
i3s

17. © 2014 Esri Suisse
Evaluation
•Configurationin GeoEventProcessoradministrator:

18. © 2014 Esri Suisse
Evaluation
•Test clientfor:
SendingMQTTmessages
Receivingi3smessages

19. © 2014 Esri Suisse
•Automaticallygenerated4 CSV filesfor:
>10 features, eachhaving10 attributes
>100 update cylces
>20%, 40%, 60%, & 80% likelihood of change in feature characteristics
Evaluation

20. © 2014 Esri Suisse
Evaluation
•DirectData Transmission
>Original CSV filesize: 275.108 bytes
>Receivedi3s filesize:1.282.878 bytes
># datapushes:1.000 (10 featuresx 100 updates)
•Pattern 1 -‘Aggregation overTime’

21. © 2014 Esri Suisse
•Pattern 2 -‘SendingLatestFeature Only’
>Withindefinedtime period, serverstoreslatestversionoffeature, whichissenttoclient
Evaluation
12% ofthedatasize
1 % of# pushes

22. © 2014 Esri Suisse
•Pattern 3 -‘SendingFeature DifferencesOnly’
>Server maintainsa mostrecentrepresentationoffeatureandsendschangedfeaturecharacteristics
Higher change likelihoods bigger data size
Evaluation
24% ofthedatasize

23. © 2014 Esri Suisse
Reference architecture+ 3 processingpatterns
>Pattern (1) ‘Aggregation over Time’
>High reduction in:# data pushes
>Good if client needs all information in periodical updates
>Pattern (2) ‘Sending only Latest Feature Representation’
>Highestreduction in: # data pushes & data size
>Best if client does not need all information
>Pattern (3) ‘Sending only Differences in Characteristics’
>High reduction in:data size
>Goodifclientneedsall information
Conclusions