The presentation introduces a Google Cloud native architecture for collecting, processing, analyzing and archiving of events from IoT devices, vehicles as well as upstream software systems. InfluxDB and its connection to global native Google Cloud services like BigQuery or Cloud Machine Learning Engine as well as Kubernetes is at the center of the architecture. The architecture demonstrates how access to global scaling cloud services addresses use cases from the Energy Sector.

1. Christoph Bussler
October 3rd, 2019
IoT Event Processing and
Analytics with InfluxDB in
Google Cloud

2. Agenda
Energy
Sector
Energy sector
use case
analysis
Life cycle
of an
IoT event
Google Cloud
1 2 3 4 5 6
TICK
architecture
Cloud native
IoT event
processing
architecture

3. ● Production
○ Energy production systems monitoring and anomaly detection
■ Oil, gas, wind, hydro, solar and others
● Distribution
○ Smart grid: maintaining an equilibrium across energy supply and demand
■ Renewable and non-renewable energy production/demand forecasting
■ https://towardsdatascience.com/how-machine-learning-can-transform-the-ener
gy-industry-caaa965e282a (devices: synchrophasers)
● Consumption
○ Fleet performance optimization (cars, trucks, planes, etc.)
○ Commercial manufacturing sites, office buildings
○ Public infrastructure
○ Private households
Use Cases: Energy Sector

4. ● Event collection
○ From production systems
○ From consumption locations
○ Absence check: requires static
equipment inventory data
● Event monitoring
○ Outages
○ Trends
○ Anomalies
■ Was there actually an outage?
Use Cases: Analysis
● Forecasting / prediction
○ Combination of
■ Current events
■ Historic events
■ Non-event data like models,
weather data, road conditions,
etc.
● Off-line analysis
○ Combination with non-event data
● Archiving
○ Long duration analysis

5. Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dynamic data
sources
Application
Systems
Short / Long
term archive

6. Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dynamic data
sources
Application
Systems
Short / Long
term archive

7. Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dynamic data
sources
Application
Systems
Short / Long
term archive

8. Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dynamic data
sources
Application
Systems
Short / Long
term archive

9. Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dynamic data
sources
Application
Systems
Short / Long
term archive

10. Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dynamic data
sources
Application
Systems
Short / Long
term archive

11. Google Cloud: Regions and Zones
https://cloud.google.com/about/locations/#regions-tab

12. Google Cloud: Global Network
https://cloud.google.com/about/locations/#network-tab

13. Google Cloud: Cloud Services
https://cloud.google.com/-select“Products”

14. Google Cloud: Select Services
BigQuery (https://cloud.google.com/bigquery/)
● Cloud native analytics database
● Columnar - SQL - large scale (PB) - multi-regional service
Cloud Spanner (https://cloud.google.com/spanner/)
● Cloud native global relational database
● Relational - SQL - consistent - linear scaling - multi-regional and intercontinental service
Coldline Storage (https://cloud.google.com/storage/docs/storage-classes)
● Cloud native storage - long term storage
Cloud Machine Learning Engine (https://cloud.google.com/ml-engine/)
● Cloud native ML engine - online and batch predictions

15. InfluxDB - TICK Architecture
https://www.influxdata.com/time-series-platform/

16. Cloud Native IoT Event Processing Architecture
Gateway

17. Cloud Native IoT Event Processing Architecture
Gateway
Cloud
IoT Core

18. Cloud Native IoT Event Processing Architecture
Gateway
Cloud
IoT Core
TICK
Stack

19. Cloud Native IoT Event Processing Architecture
Gateway
Cloud
IoT Core
TICK
Stack
Cloud
Pub/Sub
BigQuery
Cloud
Dataflow
Cloud
SQL
Cloud
Dataflow

20. Cloud Native IoT Event Processing Architecture
Gateway
Cloud
IoT Core
TICK
Stack
Cloud
Pub/Sub
BigQuery
Cloud
Dataflow
Cloud
SQL
Cloud
Dataflow
Cloud
Pub/Sub
Cloud
Spanner
Kubernetes
Engine
Cloud
Dataflow

21. Cloud Native IoT Event Processing Architecture
Gateway
Cloud
IoT Core
TICK
Stack
Cloud
Pub/Sub
BigQuery
Cloud
Dataflow
Cloud
SQL
Cloud
Dataflow
Cloud
Pub/Sub
Cloud
Spanner
Kubernetes
Engine
Cloud
Dataflow

22. Cloud Native IoT Event Processing Architecture
Gateway
Cloud
IoT Core
TICK
Stack
Cloud
Pub/Sub
BigQuery
Cloud
Dataflow
Cloud
SQL
Cloud
Pub/Sub
Cloud
Dataflow
Cloud
Pub/Sub
Cloud
Spanner
Kubernetes
Engine
Cloud Run
Cloud
Storage
AI Platform
Cloud
Dataflow

23. Cloud Native IoT Event Processing Architecture
Gateway
Cloud
IoT Core
TICK
Stack
Cloud
Pub/Sub
BigQuery
Cloud
Dataflow
Cloud
SQL
Cloud
Run
Cloud
Storage
Cloud
Pub/Sub
Cloud
Dataflow
Cloud
Pub/Sub
Cloud
Spanner
Kubernetes
Engine
Cloud
Scheduler
Cloud Run
Cloud
Storage
AI Platform
Cloud
Dataflow

24. Summary
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dynamic data
sources
Application
Systems
Short / Long
term archive

25. Thank You!

26. Q&A