Suche senden
Hochladen
Reporting: From MySQL to Hadoop/Hive
•
0 gefällt mir
•
489 views
Manuel Aldana
Folgen
Going from traditional/old-style MySQL only Reporting to Hadoop Style.
Weniger lesen
Mehr lesen
Technologie
Melden
Teilen
Melden
Teilen
1 von 42
Jetzt herunterladen
Downloaden Sie, um offline zu lesen
Empfohlen
Web 2 . .3 Development Services
Web 2 . .3 Development Services
Theawaster485
Scaling MySQL Strategies for Developers
Scaling MySQL Strategies for Developers
Jonathan Levin
Hortonworks tech workshop in-memory processing with spark
Hortonworks tech workshop in-memory processing with spark
Hortonworks
An introduction to Apache Hadoop Hive
An introduction to Apache Hadoop Hive
Mike Frampton
Scaling eBay Kleinanzeigen
Scaling eBay Kleinanzeigen
Manuel Aldana
Cross Channel Communication at eBay Kleinanzeigen
Cross Channel Communication at eBay Kleinanzeigen
Manuel Aldana
Log Monitoring at eBay kleinanzeigen (April 2013)
Log Monitoring at eBay kleinanzeigen (April 2013)
Manuel Aldana
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
Empfohlen
Web 2 . .3 Development Services
Web 2 . .3 Development Services
Theawaster485
Scaling MySQL Strategies for Developers
Scaling MySQL Strategies for Developers
Jonathan Levin
Hortonworks tech workshop in-memory processing with spark
Hortonworks tech workshop in-memory processing with spark
Hortonworks
An introduction to Apache Hadoop Hive
An introduction to Apache Hadoop Hive
Mike Frampton
Scaling eBay Kleinanzeigen
Scaling eBay Kleinanzeigen
Manuel Aldana
Cross Channel Communication at eBay Kleinanzeigen
Cross Channel Communication at eBay Kleinanzeigen
Manuel Aldana
Log Monitoring at eBay kleinanzeigen (April 2013)
Log Monitoring at eBay kleinanzeigen (April 2013)
Manuel Aldana
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Neo4j
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
Softradix Technologies
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
Radu Cotescu
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Ridwan Fadjar
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
Padma Pradeep
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Alan Dix
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Katpro Technologies
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
naman860154
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
carlostorres15106
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Mark Billinghurst
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Maria Levchenko
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
HostedbyConfluent
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
BookNet Canada
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Patryk Bandurski
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
Memoori
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Malak Abu Hammad
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
shyamraj55
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
Weitere ähnliche Inhalte
Kürzlich hochgeladen
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Neo4j
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
Softradix Technologies
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
Radu Cotescu
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Ridwan Fadjar
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
Padma Pradeep
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Alan Dix
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Katpro Technologies
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
naman860154
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
carlostorres15106
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Mark Billinghurst
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Maria Levchenko
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
HostedbyConfluent
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
BookNet Canada
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Patryk Bandurski
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
Memoori
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Malak Abu Hammad
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
shyamraj55
Kürzlich hochgeladen
(20)
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Empfohlen
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
Skeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
Introduction to Data Science
Introduction to Data Science
Christy Abraham Joy
Time Management & Productivity - Best Practices
Time Management & Productivity - Best Practices
Vit Horky
The six step guide to practical project management
The six step guide to practical project management
MindGenius
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson36
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Applitools
Empfohlen
(20)
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
Skeleton Culture Code
Skeleton Culture Code
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Getting into the tech field. what next
Getting into the tech field. what next
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
How to have difficult conversations
How to have difficult conversations
Introduction to Data Science
Introduction to Data Science
Time Management & Productivity - Best Practices
Time Management & Productivity - Best Practices
The six step guide to practical project management
The six step guide to practical project management
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Reporting: From MySQL to Hadoop/Hive
1.
Reporting: From MySQL to
Hadoop/Hive Manuel Aldana
2.
3.
Classifieds = Kleinanzeigen Ad
= Kleinanzeige
4.
April 2009 -
Sept 20092009 Launch of Discovery by DLR CC license: http://flic.kr/p/9kRijr
5.
8M Ads 11M UV
pM 1B PV pM
6.
Reporting What?
7.
http://upload.wikimedia.org/wikipedia/commons/0/0b/Relief_Map_of_Germany.svg
8.
9.
Devil / Angel
by Alper Cugun CC license: http://flic.kr/p/5tXxze
10.
Reporting back then...
11.
Batch Jobs Apache
POI 1. 2. Email 3.
12.
Problems followed...
13.
Respects of Residents
of Adenau by chuckbiscui CC license: http://flic.kr/p/eLk2o
14.
New Curbside Recycling
in Richland by Colleen Lane CC license: http://flic.kr/p/7GKY6c
15.
Sucks!!! http://flic.kr/p/7GKY6c http://www.icons-land.com
16.
Reporting Now
17.
Batch Job 1. 2. 3. pull
18.
Hive Integration
19.
Map Reduce HDFS Log-File JSON SQL
Subset (via JDBC)Batch Job nightly copyFromLocal
20.
- Model? pilot log
book by buttersweet CC license: http://flic.kr/p/8Dstb
21.
pilot log book
by buttersweet CC license: http://flic.kr/p/8Dstb
22.
23.
Batch Job 1. 2. 3. pull
24.
Why still?
25.
Tableau
26.
27.
Cats Seasonality Bikes Seasonality
28.
New Ads By
Bundesland on last Sunday
29.
Aftermath
30.
00 Respects of Residents
of Adenau by chuckbiscuito CC license: http://flic.kr/p/eLk2o
31.
New Curbside Recycling
in Richland by Colleen Lane CC license: http://flic.kr/p/7GKY6c
32.
Tug of War
by toffehoffe CC license: http://flic.kr/p/nD2nk
33.
Binoculars Portrait by
gerlos CC license: http://flic.kr/p/5KGg5B
34.
Hadoop Facts
35.
0.20.1 v. Hadoop 0.9.0
v. Hive 22 Nodes
36.
11 Reporting Jobs 1TB
Overall 5GB Daily
37.
Lessons learned
38.
http://flic.kr/p/5KGg5B pilot log book
by buttersweet CC license: http://flic.kr/p/8Dstb
39.
Log-File JSON HDFS nightly copyFromLocal
40.
Mac vs. PC
by skruk CC license: http://flic.kr/p/8sdk8Z
41.
Your lessons?
42.
Thanks!
Jetzt herunterladen