The document discusses handling real-time scenarios with IoT and Power BI. It describes using Azure services like Event Hubs, Stream Analytics, and Power BI to ingest real-time sensor data, transform it with queries, store the output in Blob storage, and display it in Power BI for real-time analytics. The presentation covers the components of an IoT solution in Azure for real-time processing and visualization of sensor data.
1. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Handling Real Time
Scenarios - IoT + Power BI
Gastón Cruz / Bruno Torterolo
MVP Data Platform / Business Technical Manager
Arkano
3. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
IoT Scenarios – Real time analytics
Today, 80% of
organizations
adopt cloud-first
strategies
AI investment
increased by
300% in 2017
Data will grow to
44 ZB in 2020
4. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
Custom
apps
Sensors
and devices
Business
apps
5. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Custom
apps
Sensors
and devices
Ingest
Business
apps
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
6. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Custom
apps
Sensors
and devices
StoreIngest
Business
apps
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
7. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Prep & Train
Custom
apps
Sensors
and devices
StoreIngest
Business
apps
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
8. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
Custom
apps
Sensors
and devices
StoreIngest
Business
apps
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
9. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
Custom
apps
Sensors
and devices
StoreIngest
Data Factory
(Data movement, pipelines & orchestration)
Business
apps
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
10. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
Custom
apps
Sensors
and devices
StoreIngest
Data Factory
(Data movement, pipelines & orchestration)
Event Hubs
IoT Hub
Business
apps
Kafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
11. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
Custom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Event Hubs
IoT Hub
Business
apps
10
01
Kafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
12. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
SparkCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Event Hubs
IoT Hub
Business
apps
10
01
Kafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
13. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
SparkCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
14. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
SparkCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
15. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
SparkCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
16. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
SparkCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
17. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
SparkCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
18. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
SparkCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Analytical dashboards
Predictive apps
Operational reports
Intelligence
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
19. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
SparkCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Analytical dashboards
Predictive apps
Operational reports
Intelligence
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
20. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
Spark
DATA
Custom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Analytical dashboards
Predictive apps
Operational reports
Intelligence
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
21. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
Spark
DATA
Custom
apps
Sensors
and devices
INTELLIGENCE
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Analytical dashboards
Predictive apps
Operational reports
Intelligence
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
22. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Model & ServePrep & Train
HDInsight /
Spark
DATA
Custom
apps
Sensors
and devices
INTELLIGENCE ACTION
Store
Blobs
Data Lake
Ingest
Data Factory
(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hubs
IoT Hub
SQL Database
Analytical dashboards
Predictive apps
Operational reports
Intelligence
Business
apps
10
01
SQLKafka
D a t a m a n a g e m e n t s o l u t i o n f o r i n t e l l i g e n c e i n t h e c l o u d
25. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
IoT Solution components
• Sensor Data - App
• Create an Azure EventHub – Ingest Data
• Create Stream Analytics Job
26. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
IoT Solution components
• Sensor Data - App
• Create an Azure EventHub – Ingest Data
• Create Stream Analytics Job
• Create Queries to Transform Real Time
27. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
IoT Solution components
• Sensor Data - App
• Create an Azure EventHub – Ingest Data
• Create Stream Analytics Job
• Create Queries to Transform Real Time
• Create Output Sinks – IE: Azure Blob Storage
28. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
IoT Solution components
• Sensor Data - App
• Create an Azure EventHub – Ingest Data
• Create Stream Analytics Job
• Create Queries to Transform Real Time
• Create Output Sinks – IE: Azure Blob Storage
• Add Power BI Output
29. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
IoT Solution components
• Sensor Data - App
• Create an Azure EventHub – Ingest Data
• Create Stream Analytics Job
• Create Queries to Transform Real Time
• Create Output Sinks – IE: Azure Blob Storage
• Add Power BI Output
• Start Stream Analytics Job
31. @gastonfcruz / @bruno_torterolo.NET Conf UY v2018
Muchas gracias! Y hasta la próxima!!!
http://www.gastoncruz.com
@gastonfcruz / @bruno_torterolo
gastoncruz@outlook.com / bruno.torterolo@arkanosoft.com
Gastón Cruz / Bruno Torterolo
Following are some examples where Azure Stream Analytics can be used:
Internet of Things(IoT) Sensor fusion and real-time analytics on device telemetry
Web logs/clickstream analytics
Geospatial analytics for fleet management and driverless vehicles
Remote monitoring and predictive maintenance of hi-value assets
Real-time analytics on Point of Sale data for inventory control and anomaly detection