Suche senden
Hochladen
HBaseCon 2013: Using Metrics to Monitor and Debug Apache HBase
•
8 gefällt mir
•
5,454 views
Cloudera, Inc.
Folgen
Presented by: Elliott Clark, Cloudera
Weniger lesen
Mehr lesen
Technologie
Sport
Melden
Teilen
Melden
Teilen
1 von 18
Empfohlen
Discoverer online training 10g r2
Discoverer online training 10g r2
Sumit Tambe
Chef Configuration Management With RightScale - RightScale Compute 2013
Chef Configuration Management With RightScale - RightScale Compute 2013
RightScale
Mastering Puppet Configuration Management in the Cloud - RightScale Compute 2013
Mastering Puppet Configuration Management in the Cloud - RightScale Compute 2013
RightScale
Topup architecture
Topup architecture
Truong Cuong
Big Data Platform Deployment Architecture
Big Data Platform Deployment Architecture
David Ko
ECE544 yashashree wase
ECE544 yashashree wase
Yashashree Wase
From Hello World to Real World - Container Days Boston 2016
From Hello World to Real World - Container Days Boston 2016
Shannon Williams
20210506 meeting2
20210506 meeting2
NickHuang49
Weitere ähnliche Inhalte
Was ist angesagt?
20210506 meeting
20210506 meeting
NickHuang49
The ultimate container monitoring bake-off - Rancher Online Meetup October 2016
The ultimate container monitoring bake-off - Rancher Online Meetup October 2016
Shannon Williams
20210506 meeting
20210506 meeting
NickHuang49
Load balancer
Load balancer
Malarvizhi Subramaniyan
Weblogic Domain Activity
Weblogic Domain Activity
subash prakash
Orchestrate Event-Driven Infrastructure with SaltStack
Orchestrate Event-Driven Infrastructure with SaltStack
Love Nyberg
Taking Akka Streams & Akka Http to Large Scale Production Applications
Taking Akka Streams & Akka Http to Large Scale Production Applications
Akara Sucharitakul
Load Balancing traffic in OpenStack neutron
Load Balancing traffic in OpenStack neutron
sufianfauzani
20210506 meeting2
20210506 meeting2
NickHuang49
Multi User Data science with Zeppelin
Multi User Data science with Zeppelin
Vinay Shukla
Naemon 2019-11-06
Naemon 2019-11-06
Sven Nierlein
20210506 meeting2
20210506 meeting2
NickHuang49
Was ist angesagt?
(12)
20210506 meeting
20210506 meeting
The ultimate container monitoring bake-off - Rancher Online Meetup October 2016
The ultimate container monitoring bake-off - Rancher Online Meetup October 2016
20210506 meeting
20210506 meeting
Load balancer
Load balancer
Weblogic Domain Activity
Weblogic Domain Activity
Orchestrate Event-Driven Infrastructure with SaltStack
Orchestrate Event-Driven Infrastructure with SaltStack
Taking Akka Streams & Akka Http to Large Scale Production Applications
Taking Akka Streams & Akka Http to Large Scale Production Applications
Load Balancing traffic in OpenStack neutron
Load Balancing traffic in OpenStack neutron
20210506 meeting2
20210506 meeting2
Multi User Data science with Zeppelin
Multi User Data science with Zeppelin
Naemon 2019-11-06
Naemon 2019-11-06
20210506 meeting2
20210506 meeting2
Andere mochten auch
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
Cloudera, Inc.
HBase Application Performance Improvement
HBase Application Performance Improvement
Biju Nair
How to Avoid Pitfalls in Big Data Analytics Webinar
How to Avoid Pitfalls in Big Data Analytics Webinar
Datameer
Spark Streaming& Kafka-The Future of Stream Processing by Hari Shreedharan of...
Spark Streaming& Kafka-The Future of Stream Processing by Hari Shreedharan of...
Data Con LA
Sqoop on Spark for Data Ingestion-(Veena Basavaraj and Vinoth Chandar, Uber)
Sqoop on Spark for Data Ingestion-(Veena Basavaraj and Vinoth Chandar, Uber)
Spark Summit
Hadoop Summit 2012 | HBase Consistency and Performance Improvements
Hadoop Summit 2012 | HBase Consistency and Performance Improvements
Cloudera, Inc.
Sqoop on Spark for Data Ingestion
Sqoop on Spark for Data Ingestion
DataWorks Summit
HBaseCon 2013: Evolving a First-Generation Apache HBase Deployment to Second...
HBaseCon 2013: Evolving a First-Generation Apache HBase Deployment to Second...
Cloudera, Inc.
HBaseCon 2013: Apache Hadoop and Apache HBase for Real-Time Video Analytics
HBaseCon 2013: Apache Hadoop and Apache HBase for Real-Time Video Analytics
Cloudera, Inc.
Tales from the Cloudera Field
Tales from the Cloudera Field
HBaseCon
HBaseCon 2012 | HBase for the Worlds Libraries - OCLC
HBaseCon 2012 | HBase for the Worlds Libraries - OCLC
Cloudera, Inc.
HBaseCon 2013: Apache HBase, Meet Ops. Ops, Meet Apache HBase.
HBaseCon 2013: Apache HBase, Meet Ops. Ops, Meet Apache HBase.
Cloudera, Inc.
HBaseCon 2015: DeathStar - Easy, Dynamic, Multi-tenant HBase via YARN
HBaseCon 2015: DeathStar - Easy, Dynamic, Multi-tenant HBase via YARN
HBaseCon
HBaseCon 2015: Trafodion - Integrating Operational SQL into HBase
HBaseCon 2015: Trafodion - Integrating Operational SQL into HBase
HBaseCon
HBase Read High Availability Using Timeline-Consistent Region Replicas
HBase Read High Availability Using Timeline-Consistent Region Replicas
HBaseCon
HBaseCon 2013: Apache HBase on Flash
HBaseCon 2013: Apache HBase on Flash
Cloudera, Inc.
Cross-Site BigTable using HBase
Cross-Site BigTable using HBase
HBaseCon
HBaseCon 2012 | Relaxed Transactions for HBase - Francis Liu, Yahoo!
HBaseCon 2012 | Relaxed Transactions for HBase - Francis Liu, Yahoo!
Cloudera, Inc.
HBaseCon 2012 | Unique Sets on HBase and Hadoop - Elliot Clark, StumbleUpon
HBaseCon 2012 | Unique Sets on HBase and Hadoop - Elliot Clark, StumbleUpon
Cloudera, Inc.
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
Cloudera, Inc.
Andere mochten auch
(20)
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBase Application Performance Improvement
HBase Application Performance Improvement
How to Avoid Pitfalls in Big Data Analytics Webinar
How to Avoid Pitfalls in Big Data Analytics Webinar
Spark Streaming& Kafka-The Future of Stream Processing by Hari Shreedharan of...
Spark Streaming& Kafka-The Future of Stream Processing by Hari Shreedharan of...
Sqoop on Spark for Data Ingestion-(Veena Basavaraj and Vinoth Chandar, Uber)
Sqoop on Spark for Data Ingestion-(Veena Basavaraj and Vinoth Chandar, Uber)
Hadoop Summit 2012 | HBase Consistency and Performance Improvements
Hadoop Summit 2012 | HBase Consistency and Performance Improvements
Sqoop on Spark for Data Ingestion
Sqoop on Spark for Data Ingestion
HBaseCon 2013: Evolving a First-Generation Apache HBase Deployment to Second...
HBaseCon 2013: Evolving a First-Generation Apache HBase Deployment to Second...
HBaseCon 2013: Apache Hadoop and Apache HBase for Real-Time Video Analytics
HBaseCon 2013: Apache Hadoop and Apache HBase for Real-Time Video Analytics
Tales from the Cloudera Field
Tales from the Cloudera Field
HBaseCon 2012 | HBase for the Worlds Libraries - OCLC
HBaseCon 2012 | HBase for the Worlds Libraries - OCLC
HBaseCon 2013: Apache HBase, Meet Ops. Ops, Meet Apache HBase.
HBaseCon 2013: Apache HBase, Meet Ops. Ops, Meet Apache HBase.
HBaseCon 2015: DeathStar - Easy, Dynamic, Multi-tenant HBase via YARN
HBaseCon 2015: DeathStar - Easy, Dynamic, Multi-tenant HBase via YARN
HBaseCon 2015: Trafodion - Integrating Operational SQL into HBase
HBaseCon 2015: Trafodion - Integrating Operational SQL into HBase
HBase Read High Availability Using Timeline-Consistent Region Replicas
HBase Read High Availability Using Timeline-Consistent Region Replicas
HBaseCon 2013: Apache HBase on Flash
HBaseCon 2013: Apache HBase on Flash
Cross-Site BigTable using HBase
Cross-Site BigTable using HBase
HBaseCon 2012 | Relaxed Transactions for HBase - Francis Liu, Yahoo!
HBaseCon 2012 | Relaxed Transactions for HBase - Francis Liu, Yahoo!
HBaseCon 2012 | Unique Sets on HBase and Hadoop - Elliot Clark, StumbleUpon
HBaseCon 2012 | Unique Sets on HBase and Hadoop - Elliot Clark, StumbleUpon
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
Mehr von Cloudera, Inc.
Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptx
Cloudera, Inc.
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists
Cloudera, Inc.
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists
Cloudera, Inc.
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019
Cloudera, Inc.
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19
Cloudera, Inc.
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Cloudera, Inc.
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Cloudera, Inc.
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Cloudera, Inc.
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Cloudera, Inc.
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
Cloudera, Inc.
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Cloudera, Inc.
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
Cloudera, Inc.
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
Cloudera, Inc.
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
Cloudera, Inc.
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
Cloudera, Inc.
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
Cloudera, Inc.
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
Cloudera, Inc.
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
Cloudera, Inc.
Mehr von Cloudera, Inc.
(20)
Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptx
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
Kürzlich hochgeladen
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
Jamie (Taka) Wang
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
David Newbury
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
D Cloud Solutions
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
DianaGray10
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
DianaGray10
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
DianaGray10
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
Udaiappa Ramachandran
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
Asko Soukka
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
Object Automation
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
shyamraj55
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
Renan Moreira de Oliveira
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
infogdgmi
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
Matt Ray
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
dgelyza
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
DianaGray10
Designing A Time bound resource download URL
Designing A Time bound resource download URL
Runcy Oommen
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
Martin Humpolec
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IES VE
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
Matsuo Lab
Kürzlich hochgeladen
(20)
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
Designing A Time bound resource download URL
Designing A Time bound resource download URL
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
HBaseCon 2013: Using Metrics to Monitor and Debug Apache HBase
1.
1 Using Metrics How to
monitor and debug Apache HBase with metrics
2.
2 Hi! I’m Elliott eclark@apache.org
@elliottnclark Engineer @ Cloudera
3.
Rules of HBase
Metrics 1. Collect Metrics 2. Collect Metrics 3. Know your steady state 4. Explore 5. Correlate 3
4.
Examples • User Error •
Region Skew • GC Pressure • Suboptimal HBase • Balancer not Balancing 4
5.
Region Skew 5 Photo
From: http://www.flickr.com/photos/64618542@N00/
6.
Region Skew 6
7.
Region Skew Symptoms •
Long tail latencies • Unbalanced WALs • Crashing region servers 7
8.
Region Skew Fixes •
Hash Row Key • Add random salt to start of Row Key • Re-Arrange The Row Key • Add Caching 8
9.
GC Pressure 9 Photo
by: http://www.flickr.com/photos/wwarby/
10.
GC Pressure 10
11.
GC Pressure Symptoms •
Crashing region servers • ZK Connection Issues • Spiky latency 11
12.
GC Pressure Fixes •
Switch map reduce to Bulk Load • Pre-Aggregate Puts • Reduce data returned per scan#next 12
13.
Bad Balancer 13 Photo
By: http://www.flickr.com/photos/jimmiehomeschoolmom/
14.
Bad Balancer 14
15.
Bad Balancer Symptoms •
Bad latency after balancer • Bad cache hit on some servers 15
16.
Coming Soon • Better
balancer • More per region metrics • Renamed Metrics • Metrics2 based metrics 16
17.
Metrics Options Cloudera Manger
OpenTSDB 17 Ganglia Ganglia Image From:http://www.flickr.com/photos/hongiiv/
18.
18
Hinweis der Redaktion
Slow DOWN!NO THUMBS!
Talk about the two different catagories.Suboptimal usage of hbaseThings hbase should handle for you.
Explain that region skew is an example.
Don’t talk about the symtoms.
Lots of hot regions on the same region serverRegions fighting over space in the Block Cache.
Go forth and collect metrics.