SlideShare ist ein Scribd-Unternehmen logo
1 von 15
1
TOPIC
2
TOPIC
Implement traffic
control system
with Big Data
Architecture
Who I am
@marcopozzan.it
www.marcopozzan.it
https://www.linkedin.com/in/marcopozzan/
Marco Pozzan
• Consultant and trainer in business intelligence, predictive analytics
• Since 2002, the main activities are related to the design of relational data warehouse and multidimensional design with
Microsoft tools.
• Since 2017 I deal with modern data warehousing and big data architectures
• Teacher at the University of Pordenone in the course of data analysis and data modelling.
• Community Lead of 1nn0va (www.innovazionefvg.net)
Agenda
Demo
Event Hub
Stream Analytics
Blob Storage
Data Lake
Service Bus
Power BI
REAL TIME ANALYTICS INFRASTRUCTURE
Lambda architecture
DEMO 1
REAL TIME ANALYTICS INFRASTRUCTURE
For the first phase of the
project, you will start building
the traffic surveillance system
to provide information on
average speeds in various
locations.
In this demo, a stream
analytics job will be created
that captures speed camera
data sent to an event hub
from a Visual Studio
application
(SpeedCameraDevice).
You will configure the a
stream analytics job to send
data to a Power BI dashboard
and Azure Data Lake
DEMO 2
REAL TIME ANALYTICS INFRASTRUCTURE
We will now add the
positions of the police
patrol cars to the traffic
surveillance system.
In this demo, a second
stream analytics job will
be created that acquires
data on the location of
patrol cars from an IoT
hub (using a Visual Studio
application,
PatrolCarDevice, to
generate the raw data).
We will configure the
Stream Analytics job to
send data to a Power BI
report and to Data Lake
DEMO 3
REAL TIME ANALYTICS INFRASTRUCTURE
The next requirement for the
traffic surveillance system is the
addition of a function for
checking vehicles registered by
speed cameras against a list of
stolen vehicles.
In this exercise, you will modify
the stream analytics job
(TrafficAnalytics) to detect if a
vehicle observed in a speed
camera has been stolen.
We will create an Azure
Storage BLOB and upload a file
containing vehicle theft records
to be used in the a stream
analytics job (in addition to
speed camera data)
DEMO 4
REAL TIME ANALYTICS INFRASTRUCTURE
Any patrol car located less
than eight kilometers from
the location most recently
reported by the stolen or
speeding vehicle could
then be dispatched to that
location.
The message must contain
the ID of the patrol car, the
registration number of the
stolen vehicle and the
coordinates of the place
where the vehicle was
observed.
In this demo, we will create
a service bus to send
warning messages to patrol
cars on stolen vehicles
Thanks
Questions?
www.marcopozzan.it @marcopozzan.it https://www.linkedin.co
m/in/marcopozzan/

Weitere ähnliche Inhalte

Was ist angesagt?

Stream processing for the practitioner: Blueprints for common stream processi...
Stream processing for the practitioner: Blueprints for common stream processi...Stream processing for the practitioner: Blueprints for common stream processi...
Stream processing for the practitioner: Blueprints for common stream processi...Aljoscha Krettek
 
batbern43 Stream all Things: Patterns of Data Integration in Event Driven Sys...
batbern43 Stream all Things: Patterns of Data Integration in Event Driven Sys...batbern43 Stream all Things: Patterns of Data Integration in Event Driven Sys...
batbern43 Stream all Things: Patterns of Data Integration in Event Driven Sys...BATbern
 
Flink Forward Berlin 2017: Bas Geerdink, Martijn Visser - Fast Data at ING - ...
Flink Forward Berlin 2017: Bas Geerdink, Martijn Visser - Fast Data at ING - ...Flink Forward Berlin 2017: Bas Geerdink, Martijn Visser - Fast Data at ING - ...
Flink Forward Berlin 2017: Bas Geerdink, Martijn Visser - Fast Data at ING - ...Flink Forward
 
ironSource Atom BigData Berlin
ironSource Atom BigData BerlinironSource Atom BigData Berlin
ironSource Atom BigData BerlinShimon Tolts
 
Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...
Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...
Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...HostedbyConfluent
 
Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...
 Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of... Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...
Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...Dataconomy Media
 
Building the Ideal Stack for Real-Time Analytics
Building the Ideal Stack for Real-Time AnalyticsBuilding the Ideal Stack for Real-Time Analytics
Building the Ideal Stack for Real-Time AnalyticsSingleStore
 
Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)
Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)
Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)Codit
 
Hardware Accelerated Machine Learning Solution for Detecting Fraud and Money ...
Hardware Accelerated Machine Learning Solution for Detecting Fraud and Money ...Hardware Accelerated Machine Learning Solution for Detecting Fraud and Money ...
Hardware Accelerated Machine Learning Solution for Detecting Fraud and Money ...TigerGraph
 
FROM DATAFRAMES TO GRAPH Data Science with pyTigerGraph
FROM DATAFRAMES TO GRAPH Data Science with pyTigerGraphFROM DATAFRAMES TO GRAPH Data Science with pyTigerGraph
FROM DATAFRAMES TO GRAPH Data Science with pyTigerGraphTigerGraph
 
User Focused Security at Netflix: Stethoscope
User Focused Security at Netflix: StethoscopeUser Focused Security at Netflix: Stethoscope
User Focused Security at Netflix: StethoscopeJesse Kriss
 
Rediscovering the Value of Apache Kafka® in Modern Data Architecture
Rediscovering the Value of Apache Kafka® in Modern Data ArchitectureRediscovering the Value of Apache Kafka® in Modern Data Architecture
Rediscovering the Value of Apache Kafka® in Modern Data Architectureconfluent
 
CTO View: Driving the On-Demand Economy with Predictive Analytics
CTO View: Driving the On-Demand Economy with Predictive AnalyticsCTO View: Driving the On-Demand Economy with Predictive Analytics
CTO View: Driving the On-Demand Economy with Predictive AnalyticsSingleStore
 
Driving the On-Demand Economy with Predictive Analytics
Driving the On-Demand Economy with Predictive AnalyticsDriving the On-Demand Economy with Predictive Analytics
Driving the On-Demand Economy with Predictive AnalyticsSingleStore
 
MemSQL - The Real-time Analytics Platform
MemSQL - The Real-time Analytics PlatformMemSQL - The Real-time Analytics Platform
MemSQL - The Real-time Analytics PlatformSingleStore
 
From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...
From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...
From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...HostedbyConfluent
 
StreamAnalytix - Multi-Engine Streaming Analytics Platform
StreamAnalytix - Multi-Engine Streaming Analytics PlatformStreamAnalytix - Multi-Engine Streaming Analytics Platform
StreamAnalytix - Multi-Engine Streaming Analytics PlatformAtul Sharma
 
Architecture Blue Print
Architecture Blue PrintArchitecture Blue Print
Architecture Blue PrintBogdan Nedelcu
 

Was ist angesagt? (20)

Stream processing for the practitioner: Blueprints for common stream processi...
Stream processing for the practitioner: Blueprints for common stream processi...Stream processing for the practitioner: Blueprints for common stream processi...
Stream processing for the practitioner: Blueprints for common stream processi...
 
batbern43 Stream all Things: Patterns of Data Integration in Event Driven Sys...
batbern43 Stream all Things: Patterns of Data Integration in Event Driven Sys...batbern43 Stream all Things: Patterns of Data Integration in Event Driven Sys...
batbern43 Stream all Things: Patterns of Data Integration in Event Driven Sys...
 
Flink Forward Berlin 2017: Bas Geerdink, Martijn Visser - Fast Data at ING - ...
Flink Forward Berlin 2017: Bas Geerdink, Martijn Visser - Fast Data at ING - ...Flink Forward Berlin 2017: Bas Geerdink, Martijn Visser - Fast Data at ING - ...
Flink Forward Berlin 2017: Bas Geerdink, Martijn Visser - Fast Data at ING - ...
 
ironSource Atom BigData Berlin
ironSource Atom BigData BerlinironSource Atom BigData Berlin
ironSource Atom BigData Berlin
 
Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...
Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...
Continuous Intelligence for Customer Service Using Kafka Event Streams | Simo...
 
Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...
 Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of... Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...
Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...
 
Building the Ideal Stack for Real-Time Analytics
Building the Ideal Stack for Real-Time AnalyticsBuilding the Ideal Stack for Real-Time Analytics
Building the Ideal Stack for Real-Time Analytics
 
Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)
Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)
Azure event hubs, Stream Analytics & Power BI (by Sam Vanhoutte)
 
Hardware Accelerated Machine Learning Solution for Detecting Fraud and Money ...
Hardware Accelerated Machine Learning Solution for Detecting Fraud and Money ...Hardware Accelerated Machine Learning Solution for Detecting Fraud and Money ...
Hardware Accelerated Machine Learning Solution for Detecting Fraud and Money ...
 
FROM DATAFRAMES TO GRAPH Data Science with pyTigerGraph
FROM DATAFRAMES TO GRAPH Data Science with pyTigerGraphFROM DATAFRAMES TO GRAPH Data Science with pyTigerGraph
FROM DATAFRAMES TO GRAPH Data Science with pyTigerGraph
 
Cap server log file analytics
Cap server log file analyticsCap server log file analytics
Cap server log file analytics
 
User Focused Security at Netflix: Stethoscope
User Focused Security at Netflix: StethoscopeUser Focused Security at Netflix: Stethoscope
User Focused Security at Netflix: Stethoscope
 
WebAction-Sami Abkay
WebAction-Sami AbkayWebAction-Sami Abkay
WebAction-Sami Abkay
 
Rediscovering the Value of Apache Kafka® in Modern Data Architecture
Rediscovering the Value of Apache Kafka® in Modern Data ArchitectureRediscovering the Value of Apache Kafka® in Modern Data Architecture
Rediscovering the Value of Apache Kafka® in Modern Data Architecture
 
CTO View: Driving the On-Demand Economy with Predictive Analytics
CTO View: Driving the On-Demand Economy with Predictive AnalyticsCTO View: Driving the On-Demand Economy with Predictive Analytics
CTO View: Driving the On-Demand Economy with Predictive Analytics
 
Driving the On-Demand Economy with Predictive Analytics
Driving the On-Demand Economy with Predictive AnalyticsDriving the On-Demand Economy with Predictive Analytics
Driving the On-Demand Economy with Predictive Analytics
 
MemSQL - The Real-time Analytics Platform
MemSQL - The Real-time Analytics PlatformMemSQL - The Real-time Analytics Platform
MemSQL - The Real-time Analytics Platform
 
From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...
From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...
From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...
 
StreamAnalytix - Multi-Engine Streaming Analytics Platform
StreamAnalytix - Multi-Engine Streaming Analytics PlatformStreamAnalytix - Multi-Engine Streaming Analytics Platform
StreamAnalytix - Multi-Engine Streaming Analytics Platform
 
Architecture Blue Print
Architecture Blue PrintArchitecture Blue Print
Architecture Blue Print
 

Ähnlich wie REAL TIME ANALYTICS INFRASTRUCTURE WITH AZURE

Rajshree R Hande Project PPT 2023 Traffic Sign Classification Using CNN and K...
Rajshree R Hande Project PPT 2023 Traffic Sign Classification Using CNN and K...Rajshree R Hande Project PPT 2023 Traffic Sign Classification Using CNN and K...
Rajshree R Hande Project PPT 2023 Traffic Sign Classification Using CNN and K...ssuser2bf502
 
Real time Analytics in IoT - Marcel Lattmann Codit Switzerland @.NET Day 2019
Real time Analytics in IoT - Marcel Lattmann Codit Switzerland @.NET Day 2019Real time Analytics in IoT - Marcel Lattmann Codit Switzerland @.NET Day 2019
Real time Analytics in IoT - Marcel Lattmann Codit Switzerland @.NET Day 2019Codit
 
Real time analytics in Azure IoT
Real time analytics in Azure IoT Real time analytics in Azure IoT
Real time analytics in Azure IoT Sam Vanhoutte
 
parking space counter [Autosaved] (2).pptx
parking space counter [Autosaved] (2).pptxparking space counter [Autosaved] (2).pptx
parking space counter [Autosaved] (2).pptxAlbertDaleSteyn
 
Rapid IoT Application Development with IBM Bluemix - Mikko Poutanen
Rapid IoT Application Development with IBM Bluemix - Mikko PoutanenRapid IoT Application Development with IBM Bluemix - Mikko Poutanen
Rapid IoT Application Development with IBM Bluemix - Mikko PoutanenWithTheBest
 
Machine Learning on dirty data - Dataiku - Forum du GFII 2014
Machine Learning on dirty data - Dataiku - Forum du GFII 2014Machine Learning on dirty data - Dataiku - Forum du GFII 2014
Machine Learning on dirty data - Dataiku - Forum du GFII 2014Le_GFII
 
React Native e IoT - Un progetto complesso
React Native e IoT - Un progetto complessoReact Native e IoT - Un progetto complesso
React Native e IoT - Un progetto complessoCommit University
 
Real-time analytics in IoT by Sam Vanhoutte (@Building The Future 2019)
Real-time analytics in IoT by Sam Vanhoutte (@Building The Future 2019)Real-time analytics in IoT by Sam Vanhoutte (@Building The Future 2019)
Real-time analytics in IoT by Sam Vanhoutte (@Building The Future 2019)Codit
 
An AI Based ATM Intelligent Security System using Open CV and YOLO
An AI Based ATM Intelligent Security System using Open CV and YOLOAn AI Based ATM Intelligent Security System using Open CV and YOLO
An AI Based ATM Intelligent Security System using Open CV and YOLOYogeshIJTSRD
 
CarStream: An Industrial System of Big Data Processing for Internet of Vehicles
CarStream: An Industrial System of Big Data Processing for Internet of VehiclesCarStream: An Industrial System of Big Data Processing for Internet of Vehicles
CarStream: An Industrial System of Big Data Processing for Internet of Vehiclesijtsrd
 
EastBanc Technologies Portals and CMS Portfolio
EastBanc Technologies Portals and CMS PortfolioEastBanc Technologies Portals and CMS Portfolio
EastBanc Technologies Portals and CMS PortfolioEastBanc Tachnologies
 
Key Imperatives for the CIO in Digital Age By Lalatendu Das Digital VP, Assoc...
Key Imperatives for the CIO in Digital Age By Lalatendu Das Digital VP, Assoc...Key Imperatives for the CIO in Digital Age By Lalatendu Das Digital VP, Assoc...
Key Imperatives for the CIO in Digital Age By Lalatendu Das Digital VP, Assoc...Rahul Neel Mani
 
Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020Riccardo Zamana
 
Azure IOT: EVENT HUB & STREAM ANALYTICS & POWER BI
 Azure IOT: EVENT HUB & STREAM ANALYTICS & POWER BI Azure IOT: EVENT HUB & STREAM ANALYTICS & POWER BI
Azure IOT: EVENT HUB & STREAM ANALYTICS & POWER BIChourouk HJAIEJ
 
ALT-F1.BE : The Accelerator (Google Cloud Platform)
ALT-F1.BE : The Accelerator (Google Cloud Platform)ALT-F1.BE : The Accelerator (Google Cloud Platform)
ALT-F1.BE : The Accelerator (Google Cloud Platform)Abdelkrim Boujraf
 

Ähnlich wie REAL TIME ANALYTICS INFRASTRUCTURE WITH AZURE (20)

Rajshree R Hande Project PPT 2023 Traffic Sign Classification Using CNN and K...
Rajshree R Hande Project PPT 2023 Traffic Sign Classification Using CNN and K...Rajshree R Hande Project PPT 2023 Traffic Sign Classification Using CNN and K...
Rajshree R Hande Project PPT 2023 Traffic Sign Classification Using CNN and K...
 
Real time Analytics in IoT - Marcel Lattmann Codit Switzerland @.NET Day 2019
Real time Analytics in IoT - Marcel Lattmann Codit Switzerland @.NET Day 2019Real time Analytics in IoT - Marcel Lattmann Codit Switzerland @.NET Day 2019
Real time Analytics in IoT - Marcel Lattmann Codit Switzerland @.NET Day 2019
 
Resume _ios
Resume _iosResume _ios
Resume _ios
 
Real time analytics in Azure IoT
Real time analytics in Azure IoT Real time analytics in Azure IoT
Real time analytics in Azure IoT
 
Cisco project ideas
Cisco   project ideasCisco   project ideas
Cisco project ideas
 
parking space counter [Autosaved] (2).pptx
parking space counter [Autosaved] (2).pptxparking space counter [Autosaved] (2).pptx
parking space counter [Autosaved] (2).pptx
 
Rapid IoT Application Development with IBM Bluemix - Mikko Poutanen
Rapid IoT Application Development with IBM Bluemix - Mikko PoutanenRapid IoT Application Development with IBM Bluemix - Mikko Poutanen
Rapid IoT Application Development with IBM Bluemix - Mikko Poutanen
 
Final paper
Final paperFinal paper
Final paper
 
Machine Learning on dirty data - Dataiku - Forum du GFII 2014
Machine Learning on dirty data - Dataiku - Forum du GFII 2014Machine Learning on dirty data - Dataiku - Forum du GFII 2014
Machine Learning on dirty data - Dataiku - Forum du GFII 2014
 
React Native e IoT - Un progetto complesso
React Native e IoT - Un progetto complessoReact Native e IoT - Un progetto complesso
React Native e IoT - Un progetto complesso
 
Real-time analytics in IoT by Sam Vanhoutte (@Building The Future 2019)
Real-time analytics in IoT by Sam Vanhoutte (@Building The Future 2019)Real-time analytics in IoT by Sam Vanhoutte (@Building The Future 2019)
Real-time analytics in IoT by Sam Vanhoutte (@Building The Future 2019)
 
Azure IOT
Azure IOTAzure IOT
Azure IOT
 
An AI Based ATM Intelligent Security System using Open CV and YOLO
An AI Based ATM Intelligent Security System using Open CV and YOLOAn AI Based ATM Intelligent Security System using Open CV and YOLO
An AI Based ATM Intelligent Security System using Open CV and YOLO
 
CarStream: An Industrial System of Big Data Processing for Internet of Vehicles
CarStream: An Industrial System of Big Data Processing for Internet of VehiclesCarStream: An Industrial System of Big Data Processing for Internet of Vehicles
CarStream: An Industrial System of Big Data Processing for Internet of Vehicles
 
EastBanc Technologies Portals and CMS Portfolio
EastBanc Technologies Portals and CMS PortfolioEastBanc Technologies Portals and CMS Portfolio
EastBanc Technologies Portals and CMS Portfolio
 
Key Imperatives for the CIO in Digital Age By Lalatendu Das Digital VP, Assoc...
Key Imperatives for the CIO in Digital Age By Lalatendu Das Digital VP, Assoc...Key Imperatives for the CIO in Digital Age By Lalatendu Das Digital VP, Assoc...
Key Imperatives for the CIO in Digital Age By Lalatendu Das Digital VP, Assoc...
 
Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020
 
Azure IOT: EVENT HUB & STREAM ANALYTICS & POWER BI
 Azure IOT: EVENT HUB & STREAM ANALYTICS & POWER BI Azure IOT: EVENT HUB & STREAM ANALYTICS & POWER BI
Azure IOT: EVENT HUB & STREAM ANALYTICS & POWER BI
 
ALT-F1.BE : The Accelerator (Google Cloud Platform)
ALT-F1.BE : The Accelerator (Google Cloud Platform)ALT-F1.BE : The Accelerator (Google Cloud Platform)
ALT-F1.BE : The Accelerator (Google Cloud Platform)
 
WSO2 ITALIA SMARTTALK #8 ASYNCAPI.pdf
WSO2 ITALIA SMARTTALK #8 ASYNCAPI.pdfWSO2 ITALIA SMARTTALK #8 ASYNCAPI.pdf
WSO2 ITALIA SMARTTALK #8 ASYNCAPI.pdf
 

Mehr von Marco Pozzan

Metadata Driven Pipeline with Microsoft Fabric
Metadata Driven Pipeline  with Microsoft FabricMetadata Driven Pipeline  with Microsoft Fabric
Metadata Driven Pipeline with Microsoft FabricMarco Pozzan
 
Data Warehouse with Fabric on data lakehouse
Data Warehouse with Fabric on data lakehouseData Warehouse with Fabric on data lakehouse
Data Warehouse with Fabric on data lakehouseMarco Pozzan
 
Data modelling for Power BI
Data modelling for Power BIData modelling for Power BI
Data modelling for Power BIMarco Pozzan
 
SlideModellingDataSat.pdf
SlideModellingDataSat.pdfSlideModellingDataSat.pdf
SlideModellingDataSat.pdfMarco Pozzan
 
Quanto mi costa SQL Pool Serverless Synapse
Quanto mi costa SQL Pool Serverless SynapseQuanto mi costa SQL Pool Serverless Synapse
Quanto mi costa SQL Pool Serverless SynapseMarco Pozzan
 
Power BI: Introduzione ai dataflow e alla preparazione dei dati self-service
Power BI: Introduzione ai dataflow e alla preparazione dei dati self-servicePower BI: Introduzione ai dataflow e alla preparazione dei dati self-service
Power BI: Introduzione ai dataflow e alla preparazione dei dati self-serviceMarco Pozzan
 
Microsoft Power BI fast with aggregation and composite model
Microsoft Power BI fast with aggregation and composite modelMicrosoft Power BI fast with aggregation and composite model
Microsoft Power BI fast with aggregation and composite modelMarco Pozzan
 
Big data analytics quanto vale e come sfruttarlo con stream analytics e power bi
Big data analytics quanto vale e come sfruttarlo con stream analytics e power biBig data analytics quanto vale e come sfruttarlo con stream analytics e power bi
Big data analytics quanto vale e come sfruttarlo con stream analytics e power biMarco Pozzan
 
What is in reality a DAX filter context
What is in reality a DAX filter contextWhat is in reality a DAX filter context
What is in reality a DAX filter contextMarco Pozzan
 
Azure saturday pn 2018
Azure saturday pn 2018Azure saturday pn 2018
Azure saturday pn 2018Marco Pozzan
 
Power B: Cleaning data
Power B: Cleaning dataPower B: Cleaning data
Power B: Cleaning dataMarco Pozzan
 
Power bi Clean and Modelling (SQL Saturday #675)
Power bi Clean and Modelling  (SQL Saturday #675)Power bi Clean and Modelling  (SQL Saturday #675)
Power bi Clean and Modelling (SQL Saturday #675)Marco Pozzan
 
Power BI and business application platform
Power BI and business application platformPower BI and business application platform
Power BI and business application platformMarco Pozzan
 

Mehr von Marco Pozzan (20)

Metadata Driven Pipeline with Microsoft Fabric
Metadata Driven Pipeline  with Microsoft FabricMetadata Driven Pipeline  with Microsoft Fabric
Metadata Driven Pipeline with Microsoft Fabric
 
Data Warehouse with Fabric on data lakehouse
Data Warehouse with Fabric on data lakehouseData Warehouse with Fabric on data lakehouse
Data Warehouse with Fabric on data lakehouse
 
Data modelling for Power BI
Data modelling for Power BIData modelling for Power BI
Data modelling for Power BI
 
SlideModellingDataSat.pdf
SlideModellingDataSat.pdfSlideModellingDataSat.pdf
SlideModellingDataSat.pdf
 
Datamart.pdf
Datamart.pdfDatamart.pdf
Datamart.pdf
 
Datamart.pptx
Datamart.pptxDatamart.pptx
Datamart.pptx
 
Quanto mi costa SQL Pool Serverless Synapse
Quanto mi costa SQL Pool Serverless SynapseQuanto mi costa SQL Pool Serverless Synapse
Quanto mi costa SQL Pool Serverless Synapse
 
Power BI: Introduzione ai dataflow e alla preparazione dei dati self-service
Power BI: Introduzione ai dataflow e alla preparazione dei dati self-servicePower BI: Introduzione ai dataflow e alla preparazione dei dati self-service
Power BI: Introduzione ai dataflow e alla preparazione dei dati self-service
 
Data flow
Data flowData flow
Data flow
 
Microsoft Power BI fast with aggregation and composite model
Microsoft Power BI fast with aggregation and composite modelMicrosoft Power BI fast with aggregation and composite model
Microsoft Power BI fast with aggregation and composite model
 
Big data analytics quanto vale e come sfruttarlo con stream analytics e power bi
Big data analytics quanto vale e come sfruttarlo con stream analytics e power biBig data analytics quanto vale e come sfruttarlo con stream analytics e power bi
Big data analytics quanto vale e come sfruttarlo con stream analytics e power bi
 
What is in reality a DAX filter context
What is in reality a DAX filter contextWhat is in reality a DAX filter context
What is in reality a DAX filter context
 
Azure saturday pn 2018
Azure saturday pn 2018Azure saturday pn 2018
Azure saturday pn 2018
 
Power B: Cleaning data
Power B: Cleaning dataPower B: Cleaning data
Power B: Cleaning data
 
Power bi Clean and Modelling (SQL Saturday #675)
Power bi Clean and Modelling  (SQL Saturday #675)Power bi Clean and Modelling  (SQL Saturday #675)
Power bi Clean and Modelling (SQL Saturday #675)
 
Power BI and business application platform
Power BI and business application platformPower BI and business application platform
Power BI and business application platform
 
Optimizing dax
Optimizing daxOptimizing dax
Optimizing dax
 
Optimizing dax
Optimizing daxOptimizing dax
Optimizing dax
 
Power bi + Flow
Power bi + FlowPower bi + Flow
Power bi + Flow
 
Power BI
Power BIPower BI
Power BI
 

Kürzlich hochgeladen

Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesZilliz
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfSeasiaInfotech2
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 

Kürzlich hochgeladen (20)

Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector Databases
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdf
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 

REAL TIME ANALYTICS INFRASTRUCTURE WITH AZURE

  • 3. Who I am @marcopozzan.it www.marcopozzan.it https://www.linkedin.com/in/marcopozzan/ Marco Pozzan • Consultant and trainer in business intelligence, predictive analytics • Since 2002, the main activities are related to the design of relational data warehouse and multidimensional design with Microsoft tools. • Since 2017 I deal with modern data warehousing and big data architectures • Teacher at the University of Pordenone in the course of data analysis and data modelling. • Community Lead of 1nn0va (www.innovazionefvg.net)
  • 4. Agenda Demo Event Hub Stream Analytics Blob Storage Data Lake Service Bus Power BI
  • 5. REAL TIME ANALYTICS INFRASTRUCTURE
  • 8. REAL TIME ANALYTICS INFRASTRUCTURE For the first phase of the project, you will start building the traffic surveillance system to provide information on average speeds in various locations. In this demo, a stream analytics job will be created that captures speed camera data sent to an event hub from a Visual Studio application (SpeedCameraDevice). You will configure the a stream analytics job to send data to a Power BI dashboard and Azure Data Lake
  • 10. REAL TIME ANALYTICS INFRASTRUCTURE We will now add the positions of the police patrol cars to the traffic surveillance system. In this demo, a second stream analytics job will be created that acquires data on the location of patrol cars from an IoT hub (using a Visual Studio application, PatrolCarDevice, to generate the raw data). We will configure the Stream Analytics job to send data to a Power BI report and to Data Lake
  • 12. REAL TIME ANALYTICS INFRASTRUCTURE The next requirement for the traffic surveillance system is the addition of a function for checking vehicles registered by speed cameras against a list of stolen vehicles. In this exercise, you will modify the stream analytics job (TrafficAnalytics) to detect if a vehicle observed in a speed camera has been stolen. We will create an Azure Storage BLOB and upload a file containing vehicle theft records to be used in the a stream analytics job (in addition to speed camera data)
  • 14. REAL TIME ANALYTICS INFRASTRUCTURE Any patrol car located less than eight kilometers from the location most recently reported by the stolen or speeding vehicle could then be dispatched to that location. The message must contain the ID of the patrol car, the registration number of the stolen vehicle and the coordinates of the place where the vehicle was observed. In this demo, we will create a service bus to send warning messages to patrol cars on stolen vehicles