Suche senden
Hochladen
Dataflow with Apache NiFi
âą
20 gefÀllt mir
âą
8,913 views
DataWorks Summit/Hadoop Summit
Folgen
Slides from the Apache NiFi CrashCourse at DataWorks Summit Munich 2017
Weniger lesen
Mehr lesen
Technologie
Melden
Teilen
Melden
Teilen
1 von 34
Jetzt herunterladen
Downloaden Sie, um offline zu lesen
Empfohlen
Apache Nifi Crash Course
Apache Nifi Crash Course
DataWorks Summit
Â
Introduction to Apache NiFi dws19 DWS - DC 2019
Introduction to Apache NiFi dws19 DWS - DC 2019
Timothy Spann
Â
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...
GetInData
Â
Data ingestion and distribution with apache NiFi
Data ingestion and distribution with apache NiFi
Lev Brailovskiy
Â
Apache NiFi in the Hadoop Ecosystem
Apache NiFi in the Hadoop Ecosystem
DataWorks Summit/Hadoop Summit
Â
Introduction to data flow management using apache nifi
Introduction to data flow management using apache nifi
Anshuman Ghosh
Â
Apache NiFi Crash Course Intro
Apache NiFi Crash Course Intro
DataWorks Summit/Hadoop Summit
Â
Running Apache NiFi with Apache Spark : Integration Options
Running Apache NiFi with Apache Spark : Integration Options
Timothy Spann
Â
Empfohlen
Apache Nifi Crash Course
Apache Nifi Crash Course
DataWorks Summit
Â
Introduction to Apache NiFi dws19 DWS - DC 2019
Introduction to Apache NiFi dws19 DWS - DC 2019
Timothy Spann
Â
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...
GetInData
Â
Data ingestion and distribution with apache NiFi
Data ingestion and distribution with apache NiFi
Lev Brailovskiy
Â
Apache NiFi in the Hadoop Ecosystem
Apache NiFi in the Hadoop Ecosystem
DataWorks Summit/Hadoop Summit
Â
Introduction to data flow management using apache nifi
Introduction to data flow management using apache nifi
Anshuman Ghosh
Â
Apache NiFi Crash Course Intro
Apache NiFi Crash Course Intro
DataWorks Summit/Hadoop Summit
Â
Running Apache NiFi with Apache Spark : Integration Options
Running Apache NiFi with Apache Spark : Integration Options
Timothy Spann
Â
Hortonworks Data in Motion Webinar Series Part 7 Apache Kafka Nifi Better Tog...
Hortonworks Data in Motion Webinar Series Part 7 Apache Kafka Nifi Better Tog...
Hortonworks
Â
Introduction to Apache NiFi 1.11.4
Introduction to Apache NiFi 1.11.4
Timothy Spann
Â
Real-time Twitter Sentiment Analysis and Image Recognition with Apache NiFi
Real-time Twitter Sentiment Analysis and Image Recognition with Apache NiFi
Timothy Spann
Â
NiFi Developer Guide
NiFi Developer Guide
Deon Huang
Â
Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose
Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose
Aldrin Piri
Â
Nifi workshop
Nifi workshop
Yifeng Jiang
Â
NiFi Best Practices for the Enterprise
NiFi Best Practices for the Enterprise
Gregory Keys
Â
Nifi
Nifi
Julio Castro
Â
Apache Nifi Crash Course
Apache Nifi Crash Course
DataWorks Summit
Â
Building Data Pipelines for Solr with Apache NiFi
Building Data Pipelines for Solr with Apache NiFi
Bryan Bende
Â
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Flink Forward
Â
Data Ingest Self Service and Management using Nifi and Kafka
Data Ingest Self Service and Management using Nifi and Kafka
DataWorks Summit
Â
Apache NiFi Meetup - Princeton NJ 2016
Apache NiFi Meetup - Princeton NJ 2016
Timothy Spann
Â
Integrating NiFi and Flink
Integrating NiFi and Flink
Bryan Bende
Â
File Format Benchmark - Avro, JSON, ORC & Parquet
File Format Benchmark - Avro, JSON, ORC & Parquet
DataWorks Summit/Hadoop Summit
Â
Securing Hadoop with Apache Ranger
Securing Hadoop with Apache Ranger
DataWorks Summit
Â
Top Three Big Data Governance Issues and How Apache ATLAS resolves it for the...
Top Three Big Data Governance Issues and How Apache ATLAS resolves it for the...
DataWorks Summit/Hadoop Summit
Â
Apache NiFi Record Processing
Apache NiFi Record Processing
Bryan Bende
Â
What's New in Apache Hive
What's New in Apache Hive
DataWorks Summit
Â
Real-Time Data Flows with Apache NiFi
Real-Time Data Flows with Apache NiFi
Manish Gupta
Â
Apache NiFi Crash Course San Jose Hadoop Summit
Apache NiFi Crash Course San Jose Hadoop Summit
Daniel Madrigal
Â
Dataflow with Apache NiFi - Crash Course - HS16SJ
Dataflow with Apache NiFi - Crash Course - HS16SJ
DataWorks Summit/Hadoop Summit
Â
Weitere Àhnliche Inhalte
Was ist angesagt?
Hortonworks Data in Motion Webinar Series Part 7 Apache Kafka Nifi Better Tog...
Hortonworks Data in Motion Webinar Series Part 7 Apache Kafka Nifi Better Tog...
Hortonworks
Â
Introduction to Apache NiFi 1.11.4
Introduction to Apache NiFi 1.11.4
Timothy Spann
Â
Real-time Twitter Sentiment Analysis and Image Recognition with Apache NiFi
Real-time Twitter Sentiment Analysis and Image Recognition with Apache NiFi
Timothy Spann
Â
NiFi Developer Guide
NiFi Developer Guide
Deon Huang
Â
Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose
Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose
Aldrin Piri
Â
Nifi workshop
Nifi workshop
Yifeng Jiang
Â
NiFi Best Practices for the Enterprise
NiFi Best Practices for the Enterprise
Gregory Keys
Â
Nifi
Nifi
Julio Castro
Â
Apache Nifi Crash Course
Apache Nifi Crash Course
DataWorks Summit
Â
Building Data Pipelines for Solr with Apache NiFi
Building Data Pipelines for Solr with Apache NiFi
Bryan Bende
Â
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Flink Forward
Â
Data Ingest Self Service and Management using Nifi and Kafka
Data Ingest Self Service and Management using Nifi and Kafka
DataWorks Summit
Â
Apache NiFi Meetup - Princeton NJ 2016
Apache NiFi Meetup - Princeton NJ 2016
Timothy Spann
Â
Integrating NiFi and Flink
Integrating NiFi and Flink
Bryan Bende
Â
File Format Benchmark - Avro, JSON, ORC & Parquet
File Format Benchmark - Avro, JSON, ORC & Parquet
DataWorks Summit/Hadoop Summit
Â
Securing Hadoop with Apache Ranger
Securing Hadoop with Apache Ranger
DataWorks Summit
Â
Top Three Big Data Governance Issues and How Apache ATLAS resolves it for the...
Top Three Big Data Governance Issues and How Apache ATLAS resolves it for the...
DataWorks Summit/Hadoop Summit
Â
Apache NiFi Record Processing
Apache NiFi Record Processing
Bryan Bende
Â
What's New in Apache Hive
What's New in Apache Hive
DataWorks Summit
Â
Real-Time Data Flows with Apache NiFi
Real-Time Data Flows with Apache NiFi
Manish Gupta
Â
Was ist angesagt?
(20)
Hortonworks Data in Motion Webinar Series Part 7 Apache Kafka Nifi Better Tog...
Hortonworks Data in Motion Webinar Series Part 7 Apache Kafka Nifi Better Tog...
Â
Introduction to Apache NiFi 1.11.4
Introduction to Apache NiFi 1.11.4
Â
Real-time Twitter Sentiment Analysis and Image Recognition with Apache NiFi
Real-time Twitter Sentiment Analysis and Image Recognition with Apache NiFi
Â
NiFi Developer Guide
NiFi Developer Guide
Â
Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose
Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose
Â
Nifi workshop
Nifi workshop
Â
NiFi Best Practices for the Enterprise
NiFi Best Practices for the Enterprise
Â
Nifi
Nifi
Â
Apache Nifi Crash Course
Apache Nifi Crash Course
Â
Building Data Pipelines for Solr with Apache NiFi
Building Data Pipelines for Solr with Apache NiFi
Â
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Â
Data Ingest Self Service and Management using Nifi and Kafka
Data Ingest Self Service and Management using Nifi and Kafka
Â
Apache NiFi Meetup - Princeton NJ 2016
Apache NiFi Meetup - Princeton NJ 2016
Â
Integrating NiFi and Flink
Integrating NiFi and Flink
Â
File Format Benchmark - Avro, JSON, ORC & Parquet
File Format Benchmark - Avro, JSON, ORC & Parquet
Â
Securing Hadoop with Apache Ranger
Securing Hadoop with Apache Ranger
Â
Top Three Big Data Governance Issues and How Apache ATLAS resolves it for the...
Top Three Big Data Governance Issues and How Apache ATLAS resolves it for the...
Â
Apache NiFi Record Processing
Apache NiFi Record Processing
Â
What's New in Apache Hive
What's New in Apache Hive
Â
Real-Time Data Flows with Apache NiFi
Real-Time Data Flows with Apache NiFi
Â
Ăhnlich wie Dataflow with Apache NiFi
Apache NiFi Crash Course San Jose Hadoop Summit
Apache NiFi Crash Course San Jose Hadoop Summit
Daniel Madrigal
Â
Dataflow with Apache NiFi - Crash Course - HS16SJ
Dataflow with Apache NiFi - Crash Course - HS16SJ
DataWorks Summit/Hadoop Summit
Â
Intelligently Collecting Data at the Edge - Intro to Apache MiNiFi
Intelligently Collecting Data at the Edge - Intro to Apache MiNiFi
DataWorks Summit
Â
Apache NiFi Crash Course - San Jose Hadoop Summit
Apache NiFi Crash Course - San Jose Hadoop Summit
Aldrin Piri
Â
The Avant-garde of Apache NiFi
The Avant-garde of Apache NiFi
DataWorks Summit/Hadoop Summit
Â
The Avant-garde of Apache NiFi
The Avant-garde of Apache NiFi
Joe Percivall
Â
[Hortonworks] Future Of Data: Madrid - HDF & Data in motion
[Hortonworks] Future Of Data: Madrid - HDF & Data in motion
RaĂșl MarĂn
Â
Connecting the Drops with Apache NiFi & Apache MiNiFi
Connecting the Drops with Apache NiFi & Apache MiNiFi
DataWorks Summit
Â
Apache Nifi Crash Course
Apache Nifi Crash Course
DataWorks Summit
Â
Intelligently collecting data at the edgeâintro to Apache MiNiFi
Intelligently collecting data at the edgeâintro to Apache MiNiFi
DataWorks Summit
Â
Data at Scales and the Values of Starting Small with Apache NiFi & MiNiFi
Data at Scales and the Values of Starting Small with Apache NiFi & MiNiFi
Aldrin Piri
Â
Using ApacheÂź NiFi to Empower Self-Organising Teams
Using ApacheÂź NiFi to Empower Self-Organising Teams
Sebastian Carroll
Â
Apache NiFi + Tensorflow + HadoopïŒBig Data AI ă”ăłăă€ăăăźäœăæč
Apache NiFi + Tensorflow + HadoopïŒBig Data AI ă”ăłăă€ăăăźäœăæč
HortonworksJapan
Â
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Hortonworks
Â
Apache Zeppelin + LIvy: Bringing Multi Tenancy to Interactive Data Analysis
Apache Zeppelin + LIvy: Bringing Multi Tenancy to Interactive Data Analysis
DataWorks Summit/Hadoop Summit
Â
Hadoop Summit Tokyo Apache NiFi Crash Course
Hadoop Summit Tokyo Apache NiFi Crash Course
DataWorks Summit/Hadoop Summit
Â
Introduction to Apache NiFi - Seattle Scalability Meetup
Introduction to Apache NiFi - Seattle Scalability Meetup
Saptak Sen
Â
Big Data Day LA 2016/ Big Data Track - Building scalable enterprise data flow...
Big Data Day LA 2016/ Big Data Track - Building scalable enterprise data flow...
Data Con LA
Â
HDF Powered by Apache NiFi Introduction
HDF Powered by Apache NiFi Introduction
Milind Pandit
Â
Forget Duplicating Local Changes: Apache NiFi and the Flow Development Lifecy...
Forget Duplicating Local Changes: Apache NiFi and the Flow Development Lifecy...
DataWorks Summit
Â
Ăhnlich wie Dataflow with Apache NiFi
(20)
Apache NiFi Crash Course San Jose Hadoop Summit
Apache NiFi Crash Course San Jose Hadoop Summit
Â
Dataflow with Apache NiFi - Crash Course - HS16SJ
Dataflow with Apache NiFi - Crash Course - HS16SJ
Â
Intelligently Collecting Data at the Edge - Intro to Apache MiNiFi
Intelligently Collecting Data at the Edge - Intro to Apache MiNiFi
Â
Apache NiFi Crash Course - San Jose Hadoop Summit
Apache NiFi Crash Course - San Jose Hadoop Summit
Â
The Avant-garde of Apache NiFi
The Avant-garde of Apache NiFi
Â
The Avant-garde of Apache NiFi
The Avant-garde of Apache NiFi
Â
[Hortonworks] Future Of Data: Madrid - HDF & Data in motion
[Hortonworks] Future Of Data: Madrid - HDF & Data in motion
Â
Connecting the Drops with Apache NiFi & Apache MiNiFi
Connecting the Drops with Apache NiFi & Apache MiNiFi
Â
Apache Nifi Crash Course
Apache Nifi Crash Course
Â
Intelligently collecting data at the edgeâintro to Apache MiNiFi
Intelligently collecting data at the edgeâintro to Apache MiNiFi
Â
Data at Scales and the Values of Starting Small with Apache NiFi & MiNiFi
Data at Scales and the Values of Starting Small with Apache NiFi & MiNiFi
Â
Using ApacheÂź NiFi to Empower Self-Organising Teams
Using ApacheÂź NiFi to Empower Self-Organising Teams
Â
Apache NiFi + Tensorflow + HadoopïŒBig Data AI ă”ăłăă€ăăăźäœăæč
Apache NiFi + Tensorflow + HadoopïŒBig Data AI ă”ăłăă€ăăăźäœăæč
Â
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Â
Apache Zeppelin + LIvy: Bringing Multi Tenancy to Interactive Data Analysis
Apache Zeppelin + LIvy: Bringing Multi Tenancy to Interactive Data Analysis
Â
Hadoop Summit Tokyo Apache NiFi Crash Course
Hadoop Summit Tokyo Apache NiFi Crash Course
Â
Introduction to Apache NiFi - Seattle Scalability Meetup
Introduction to Apache NiFi - Seattle Scalability Meetup
Â
Big Data Day LA 2016/ Big Data Track - Building scalable enterprise data flow...
Big Data Day LA 2016/ Big Data Track - Building scalable enterprise data flow...
Â
HDF Powered by Apache NiFi Introduction
HDF Powered by Apache NiFi Introduction
Â
Forget Duplicating Local Changes: Apache NiFi and the Flow Development Lifecy...
Forget Duplicating Local Changes: Apache NiFi and the Flow Development Lifecy...
Â
Mehr von DataWorks Summit/Hadoop Summit
Running Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in Production
DataWorks Summit/Hadoop Summit
Â
State of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache Zeppelin
DataWorks Summit/Hadoop Summit
Â
Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache Ranger
DataWorks Summit/Hadoop Summit
Â
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science Platform
DataWorks Summit/Hadoop Summit
Â
Revolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and Zeppelin
DataWorks Summit/Hadoop Summit
Â
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSense
DataWorks Summit/Hadoop Summit
Â
Hadoop Crash Course
Hadoop Crash Course
DataWorks Summit/Hadoop Summit
Â
Data Science Crash Course
Data Science Crash Course
DataWorks Summit/Hadoop Summit
Â
Apache Spark Crash Course
Apache Spark Crash Course
DataWorks Summit/Hadoop Summit
Â
Schema Registry - Set you Data Free
Schema Registry - Set you Data Free
DataWorks Summit/Hadoop Summit
Â
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
DataWorks Summit/Hadoop Summit
Â
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
DataWorks Summit/Hadoop Summit
Â
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and ML
DataWorks Summit/Hadoop Summit
Â
How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient
DataWorks Summit/Hadoop Summit
Â
HBase in Practice
HBase in Practice
DataWorks Summit/Hadoop Summit
Â
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)
DataWorks Summit/Hadoop Summit
Â
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
DataWorks Summit/Hadoop Summit
Â
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
DataWorks Summit/Hadoop Summit
Â
Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop
DataWorks Summit/Hadoop Summit
Â
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
DataWorks Summit/Hadoop Summit
Â
Mehr von DataWorks Summit/Hadoop Summit
(20)
Running Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in Production
Â
State of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache Zeppelin
Â
Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache Ranger
Â
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science Platform
Â
Revolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and Zeppelin
Â
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSense
Â
Hadoop Crash Course
Hadoop Crash Course
Â
Data Science Crash Course
Data Science Crash Course
Â
Apache Spark Crash Course
Apache Spark Crash Course
Â
Schema Registry - Set you Data Free
Schema Registry - Set you Data Free
Â
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Â
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Â
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and ML
Â
How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient
Â
HBase in Practice
HBase in Practice
Â
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)
Â
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Â
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
Â
Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop
Â
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Scaling HDFS to Manage Billions of Files with Distributed Storage Schemes
Â
KĂŒrzlich hochgeladen
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
Â
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
Â
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
MIND CTI
Â
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
Khem
Â
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Principled Technologies
Â
Scaling API-first â The story of a global engineering organization
Scaling API-first â The story of a global engineering organization
Radu Cotescu
Â
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Edi Saputra
Â
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
Boston Institute of Analytics
Â
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
The Digital Insurer
Â
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Drew Madelung
Â
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
Â
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
apidays
Â
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
Product Anonymous
Â
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
apidays
Â
đŹ The future of MySQL is Postgres đ
đŹ The future of MySQL is Postgres đ
RTylerCroy
Â
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
Â
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc
Â
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
Â
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Roshan Dwivedi
Â
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
Â
KĂŒrzlich hochgeladen
(20)
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Â
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Â
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
Â
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
Â
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Â
Scaling API-first â The story of a global engineering organization
Scaling API-first â The story of a global engineering organization
Â
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Â
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
Â
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
Â
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Â
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Â
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Â
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
Â
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Â
đŹ The future of MySQL is Postgres đ
đŹ The future of MySQL is Postgres đ
Â
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
Â
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
Â
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Â
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Â
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Â
Dataflow with Apache NiFi
1.
Dataflow with Apache NiFi Aldrin Piri - @aldrinpiri Apache NiFi Crash Course DataWorks
Summit 2017 â Munich 6 April 2017
2.
2 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Key: 'Apache NiFiâ Value: 'PMC Member' Key: 'Workâ Value: âSr. Member of Technical Staff @ Hortonworks' Key: 'Working with NiFi
Sinceâ Value: '2010â
3.
3 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Agenda What is dataflow and what are the challenges? Apache NiFi Architecture Live Demo Community
4.
4 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Agenda What is dataflow and what are the challenges? Apache NiFi Architecture Live Demo Community
5.
5 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Letâs Connect A to B Producers A.K.A Things Anything AND Everything Internet! Consumers âą
User âą Storage âą System âą âŠMore Things
6.
6 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Moving data effectively is hard Standards: http://xkcd.com/927/
7.
7 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Why is moving data effectively hard? Ă
Standards Ă Formats Ă âExactly Onceâ Delivery Ă Protocols Ă Veracity of Information Ă Validity of Information Ă Ensuring Security Ă Overcoming Security Ă Compliance Ă Schemas Ă Consumers Change Ă Credential Management Ă âThat [person|team|group]â Ă Network Ă âExactly Onceâ Delivery
8.
8 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Letâs Connect Lots of As to Bs
to As to Cs to Bs to Îs to Cs to Ïs Letâs consider the needs of a courier service Physical Store Gateway Server Mobile Devices Registers Server Cluster Distribution Center Core Data Center at HQ Server Cluster On Delivery Routes Trucks Deliverers Delivery Truck: Creative Stall, https://thenounproject.com/creativestall/ Deliverer: Rigo Peter, https://thenounproject.com/rigo/ Cash Register: Sergey Patutin, https://thenounproject.com/bdesign.by/ Hand Scanner: Eric Pearson, https://thenounproject.com/epearson001/
9.
9 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Great! I am collecting all this data! Letâs use it! Finding our needles in the haystack Physical Store Gateway Server Mobile Devices Registers Server Cluster Distribution Center Kafka Core Data Center at HQ Server Cluster Others Storm / Spark / Flink
/ Apex Kafka Storm / Spark / Flink / Apex On Delivery Routes Trucks Deliverers Delivery Truck: Creative Stall, https://thenounproject.com/creativestall/ Deliverer: Rigo Peter, https://thenounproject.com/rigo/ Cash Register: Sergey Patutin, https://thenounproject.com/bdesign.by/ Hand Scanner: Eric Pearson, https://thenounproject.com/epearson001/
10.
10 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Why is moving data effectively hard when scoped internally? Ă
Standards Ă Formats Ă âExactly Onceâ Delivery Ă Protocols Ă Veracity of Information Ă Validity of Information Ă Ensuring Security Ă Overcoming Security Ă Compliance Ă Schemas Ă Consumers Change Ă Credential Management Ă âThat [person|team|group]â Ă Network Ă âExactly Onceâ Delivery
11.
11 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Letâs Connect Lots of As to Bs
to As to Cs to Bs to Îs to Cs to Ïs Oh, that courier service is global
12.
12 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Why is moving data effectively hard when scoped globally? Ă
Standards Ă Formats Ă âExactly Onceâ Delivery Ă Protocols Ă Veracity of Information Ă Validity of Information Ă Ensuring Security Ă Overcoming Security Ă Compliance Ă Schemas Ă Consumers Change Ă Credential Management Ă âThat [person|team|group]â Ă Network Ă âExactly Onceâ Delivery
13.
13 © Hortonworks Inc. 2011 â 2016. All Rights Reserved The Unassuming Line: A Case Study Weâve seen a few lines show up in the wild thus far Internet!
Inter- & Intra- connections in our global courier enterprise Spotlight: Arthur LacĂŽte, https://thenounproject.com/turo/
14.
14 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Dataflow Line Anatomy 101 Letâs dissect what this line typically represents Fig 1. Lineus
Worldwidewebus. Common Name: Internet! Script or Application Script or Application Data Data Disparate Transport Mechanisms
15.
15 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Dataflow Line Anatomy 201 Sometimes that transport is just more lines Fig 1. Lineus
Worldwidewebus. Common Name: Internet! Script or Application Script or Application Line Inception Data Data
16.
16 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Dataflow Line Anatomy 301 But those lines could also have components⊠Fig 1. Lineus
Worldwidewebus. Common Name: Internet!
17.
17 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Agenda What is dataflow and what are the challenges? Apache NiFi Architecture Live Demo Community
18.
18 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Apache NiFi Key Features âą
Guaranteed delivery âą Data buffering - Backpressure - Pressure release âą Prioritized queuing âą Flow specific QoS - Latency vs. throughput - Loss tolerance âą Data provenance âą Supports push and pull models âą Recovery/recording a rolling log of fine- grained history âą Visual command and control âą Flow templates âą Pluggable/multi-role security âą Designed for extension âą Clustering
19.
19 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Apache NiFi
Subproject: MiNiFi Ă Let me get the key parts of NiFi close to where data begins and provide bidrectional communication Ă NiFi lives in the data center. Give it an enterprise server or a cluster of them. Ă MiNiFi lives as close to where data is born and is a guest on that device or system
20.
20 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Letâs revisit our courier service from the perspective of NiFi Physical Store Gateway Server Mobile Devices Registers Server Cluster Distribution Center Kafka Core Data Center at HQ Server Cluster Others Storm / Spark / Flink
/ Apex Kafka Storm / Spark / Flink / Apex On Delivery Routes Trucks Deliverers Delivery Truck: Creative Stall, https://thenounproject.com/creativestall/ Deliverer: Rigo Peter, https://thenounproject.com/rigo/ Cash Register: Sergey Patutin, https://thenounproject.com/bdesign.by/ Hand Scanner: Eric Pearson, https://thenounproject.com/epearson001/ Client Libraries Client Libraries MiNiFi MiNiFi NiFi NiFi NiFi NiFi NiFi NiFi Client Libraries
21.
21 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Apache NiFi
Managed Dataflow SOURCES REGIONAL INFRASTRUCTURE CORE INFRASTRUCTURE
22.
22 © Hortonworks Inc. 2011 â 2016. All Rights Reserved NiFi
is based on Flow Based Programming (FBP) FBP Term NiFi Term Description Information Packet FlowFile Each object moving through the system. Black Box FlowFile Processor Performs the work, doing some combination of data routing, transformation, or mediation between systems. Bounded Buffer Connection The linkage between processors, acting as queues and allowing various processes to interact at differing rates. Scheduler Flow Controller Maintains the knowledge of how processes are connected, and manages the threads and allocations thereof which all processes use. Subnet Process Group A set of processes and their connections, which can receive and send data via ports. A process group allows creation of entirely new component simply by composition of its components.
23.
23 © Hortonworks Inc. 2011 â 2016. All Rights Reserved FlowFiles
& Data Agnosticism Ă NiFi is data agnostic! Ă But, NiFi was designed understanding that users can care about specifics and provides tooling to interact with specific formats, protocols, etc. ISO 8601 - http://xkcd.com/1179/ Robustness principle Be conservative in what you do, be liberal in what you accept from othersâ
24.
24 © Hortonworks Inc. 2011 â 2016. All Rights Reserved FlowFiles
are like HTTP data HTTP Data FlowFile HTTP/1.1 200 OK Date: Sun, 10 Oct 2010 23:26:07 GMT Server: Apache/2.2.8 (CentOS) OpenSSL/0.9.8g Last-Modified: Sun, 26 Sep 2010 22:04:35 GMT ETag: "45b6-834-49130cc1182c0" Accept-Ranges: bytes Content-Length: 13 Connection: close Content-Type: text/html Hello world! Standard FlowFile Attributes Key: 'entryDateâ Value: 'Fri Jun 17 17:15:04 EDT 2016' Key: 'lineageStartDateâ Value: 'Fri Jun 17 17:15:04 EDT 2016' Key: 'fileSizeâ Value: '23609' FlowFile Attribute Map Content Key: 'filenameâ Value: '15650246997242' Key: 'pathâ Value: './â Binary Content * Header Content
25.
25 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Agenda What is dataflow and what are the challenges? Apache NiFi Architecture Live Demo Community
26.
26 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Extension / Integration Points NiFi
Term Description Flow File Processor Push/Pull behavior. Custom UI Reporting Task Used to push data from NiFi to some external service (metrics, provenance, etc..) Controller Service Used to enable reusable components / shared services throughout the flow REST API Allows clients to connect to pull information, change behavior, etc.. © Hortonworks Inc. 2011 â 2016. All Rights ReservedX Architecture OS/Host JVM Flow Controller Web Server Processor 1 Extension N FlowFileâš Repository Contentâš Repository Provenanceâš Repository Local Storage Standalone Cluster
27.
27 © Hortonworks Inc. 2011 â 2016. All Rights Reserved NiFi Architecture â
Repositories - Pass by reference FlowFile Content Provenance F1Ă C1 C1 P1Ă F1 Excerpt of demo flow⊠Whatâs happening inside the repositories⊠BEFORE AFTER F2Ă C1 C1 P3Ă F2 â Clone (F1) F1Ă C1 P2Ă F1 â Route P1Ă F1 â Create
28.
28 © Hortonworks Inc. 2011 â 2016. All Rights Reserved NiFi Architecture â
Repositories â Copy on Write FlowFile Content Provenance F1Ă C1 C1 P1Ă F1 - CREATE Excerpt of demo flow⊠Whatâs happening inside the repositories⊠BEFORE AFTER F1Ă C1 F1.1Ă C2 C2 (encrypted) C1 (plaintext) P2Ă F1.1 - MODIFY P1Ă F1 - CREATE
29.
29 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Agenda What is dataflow and what are the challenges? Apache NiFi Architecture Demo Community
30.
30 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Learn, Share at Birds of a Feather IOT, STREAMING & DATA FLOW Thursday, April 6 5:50 pm, Room 5
31.
31 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Why NiFi? Ă
Moving data is multifaceted in its challenges and these are present in different contexts at varying scopes â Think of our courier example and organizations like it: inter vs intra, domestically, internationally Ă Provide common tooling and extensions that are commonly needed but be flexible for extension â Leverage existing libraries and expansive Java ecosystem for functionality â Allow organizations to integrate with their existing infrastructure Ă Empower folks managing your infrastructure to make changes and reason about issues that are occurring â Data Provenance to show context and dataâs journey â User Interface/Experience a key component
32.
32 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Learn more and join us! Apache
NiFi site http://nifi.apache.org Subproject MiNiFi site http://nifi.apache.org/minifi/ Subscribe to and collaborate at dev@nifi.apache.org users@nifi.apache.org Submit Ideas or Issues https://issues.apache.org/jira/browse/NIFI Follow us on Twitter @apachenifi
33.
33 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Our Lab for Today Ă
We will be exploring some examples to work through creating a dataflow with Apache NiFi Ă Use Case: An urban planning board is evaluating the need for a new highway, dependent on current traffic patterns, particularly as other roadwork initiatives are under way. Integrating live data poses a problem because traffic analysis has traditionally been done using historical, aggregated traffic counts. To improve traffic analysis, the city planner wants to leverage real-time data to get a deeper understanding of traffic patterns. NiFi was selected for for this real-time data integration. Ă Labs are available at http://tinyurl.com/nificrashcourse
34.
34 © Hortonworks Inc. 2011 â 2016. All Rights Reserved Thank You
Jetzt herunterladen