SlideShare ist ein Scribd-Unternehmen logo

openTSDB - Metrics for a distributed world

Oliver Hankeln
Oliver Hankeln
Oliver HankelnFreelance IT Consultant um Hankeln Consulting

These are the slides for my talk at the IPC13/WTC13 in Munich on openTSDB. openTSDB ist the software that we at gutefrage.net use to store about 200 million data points in several thousand time series per day. I will talk about how openTSDB stores the data to efficiently query them afterwards. Some cultural issues and some myths are also covered.

openTSDB - Metrics for a distributed world

1 von 46
Downloaden Sie, um offline zu lesen
openTSDB - Metrics for
a distributed world
Oliver Hankeln / gutefrage.net
@mydalon

Mittwoch, 30. Oktober 13
Who am I?
Senior Engineer - Data and Infrastructure at
gutefrage.net GmbH
Was doing software development before
DevOps advocate

Mittwoch, 30. Oktober 13
Who is Gutefrage.net?
Germany‘s biggest Q&A platform
#1 German site (mobile) about 5M Unique Users
#3 German site (desktop) about 17M Unique Users
> 4 Mio PI/day

Part of the Holtzbrinck group
Running several platforms (Gutefrage.net,
Helpster.de, Cosmiq, Comprano, ...)

Mittwoch, 30. Oktober 13
What you will get
Why we chose openTSDB
What is openTSDB?
How does openTSDB store the data?
Our experiences
Some advice

Mittwoch, 30. Oktober 13
Why we chose
openTSDB

Mittwoch, 30. Oktober 13
We were looking at
some options
Munin

Graphite openTSDB

Ganglia

Scales
well

no

sort of

yes

yes

Keeps all
data

no

no

yes

no

Creating
metrics

easy

easy

easy

easy

Mittwoch, 30. Oktober 13
Anzeige

Recomendados

OpenTSDB 2.0
OpenTSDB 2.0OpenTSDB 2.0
OpenTSDB 2.0HBaseCon
 
OpenTSDB for monitoring @ Criteo
OpenTSDB for monitoring @ CriteoOpenTSDB for monitoring @ Criteo
OpenTSDB for monitoring @ CriteoNathaniel Braun
 
Ted Dunning – Very High Bandwidth Time Series Database Implementation - NoSQL...
Ted Dunning – Very High Bandwidth Time Series Database Implementation - NoSQL...Ted Dunning – Very High Bandwidth Time Series Database Implementation - NoSQL...
Ted Dunning – Very High Bandwidth Time Series Database Implementation - NoSQL...NoSQLmatters
 
HBaseCon 2012 | Lessons learned from OpenTSDB - Benoit Sigoure, StumbleUpon
HBaseCon 2012 | Lessons learned from OpenTSDB - Benoit Sigoure, StumbleUponHBaseCon 2012 | Lessons learned from OpenTSDB - Benoit Sigoure, StumbleUpon
HBaseCon 2012 | Lessons learned from OpenTSDB - Benoit Sigoure, StumbleUponCloudera, Inc.
 
OpenTSDB: HBaseCon2017
OpenTSDB: HBaseCon2017OpenTSDB: HBaseCon2017
OpenTSDB: HBaseCon2017HBaseCon
 
Update on OpenTSDB and AsyncHBase
Update on OpenTSDB and AsyncHBase Update on OpenTSDB and AsyncHBase
Update on OpenTSDB and AsyncHBase HBaseCon
 
Bucket your partitions wisely - Cassandra summit 2016
Bucket your partitions wisely - Cassandra summit 2016Bucket your partitions wisely - Cassandra summit 2016
Bucket your partitions wisely - Cassandra summit 2016Markus Höfer
 
Bucket Your Partitions Wisely (Markus Höfer, codecentric AG) | Cassandra Summ...
Bucket Your Partitions Wisely (Markus Höfer, codecentric AG) | Cassandra Summ...Bucket Your Partitions Wisely (Markus Höfer, codecentric AG) | Cassandra Summ...
Bucket Your Partitions Wisely (Markus Höfer, codecentric AG) | Cassandra Summ...DataStax
 

Más contenido relacionado

Was ist angesagt?

FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...
FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...
FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...Rob Skillington
 
Go and Uber’s time series database m3
Go and Uber’s time series database m3Go and Uber’s time series database m3
Go and Uber’s time series database m3Rob Skillington
 
Time Series Data in a Time Series World
Time Series Data in a Time Series WorldTime Series Data in a Time Series World
Time Series Data in a Time Series WorldMapR Technologies
 
Apache Solr as a compressed, scalable, and high performance time series database
Apache Solr as a compressed, scalable, and high performance time series databaseApache Solr as a compressed, scalable, and high performance time series database
Apache Solr as a compressed, scalable, and high performance time series databaseFlorian Lautenschlager
 
Pain points with M3, some things to address them and how replication works
Pain points with M3, some things to address them and how replication worksPain points with M3, some things to address them and how replication works
Pain points with M3, some things to address them and how replication worksRob Skillington
 
FOSDEM 2020: Querying over millions and billions of metrics with M3DB's index
FOSDEM 2020: Querying over millions and billions of metrics with M3DB's indexFOSDEM 2020: Querying over millions and billions of metrics with M3DB's index
FOSDEM 2020: Querying over millions and billions of metrics with M3DB's indexRob Skillington
 
Advanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXAdvanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXzznate
 
Chronix Poster for the Poster Session FAST 2017
Chronix Poster for the Poster Session FAST 2017Chronix Poster for the Poster Session FAST 2017
Chronix Poster for the Poster Session FAST 2017Florian Lautenschlager
 
A Fast and Efficient Time Series Storage Based on Apache Solr
A Fast and Efficient Time Series Storage Based on Apache SolrA Fast and Efficient Time Series Storage Based on Apache Solr
A Fast and Efficient Time Series Storage Based on Apache SolrQAware GmbH
 
Efficient and Fast Time Series Storage - The missing link in dynamic software...
Efficient and Fast Time Series Storage - The missing link in dynamic software...Efficient and Fast Time Series Storage - The missing link in dynamic software...
Efficient and Fast Time Series Storage - The missing link in dynamic software...Florian Lautenschlager
 
Performance evaluation of apache tajo
Performance evaluation of apache tajoPerformance evaluation of apache tajo
Performance evaluation of apache tajoJihoon Son
 
Discretized Stream - Fault-Tolerant Streaming Computation at Scale - SOSP
Discretized Stream - Fault-Tolerant Streaming Computation at Scale - SOSPDiscretized Stream - Fault-Tolerant Streaming Computation at Scale - SOSP
Discretized Stream - Fault-Tolerant Streaming Computation at Scale - SOSPTathagata Das
 
Data Structures and Performance for Scientific Computing with Hadoop and Dumb...
Data Structures and Performance for Scientific Computing with Hadoop and Dumb...Data Structures and Performance for Scientific Computing with Hadoop and Dumb...
Data Structures and Performance for Scientific Computing with Hadoop and Dumb...Austin Benson
 
OSDC 2016 - Chronix - A fast and efficient time series storage based on Apach...
OSDC 2016 - Chronix - A fast and efficient time series storage based on Apach...OSDC 2016 - Chronix - A fast and efficient time series storage based on Apach...
OSDC 2016 - Chronix - A fast and efficient time series storage based on Apach...NETWAYS
 
ToroDB: scaling PostgreSQL like MongoDB / Álvaro Hernández Tortosa (8Kdata)
ToroDB: scaling PostgreSQL like MongoDB / Álvaro Hernández Tortosa (8Kdata)ToroDB: scaling PostgreSQL like MongoDB / Álvaro Hernández Tortosa (8Kdata)
ToroDB: scaling PostgreSQL like MongoDB / Álvaro Hernández Tortosa (8Kdata)Ontico
 
Query optimization in Apache Tajo
Query optimization in Apache TajoQuery optimization in Apache Tajo
Query optimization in Apache TajoJihoon Son
 
Gnocchi v4 (preview)
Gnocchi v4 (preview)Gnocchi v4 (preview)
Gnocchi v4 (preview)Gordon Chung
 
RDO hangout on gnocchi
RDO hangout on gnocchiRDO hangout on gnocchi
RDO hangout on gnocchiEoghan Glynn
 

Was ist angesagt? (20)

FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...
FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...
FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...
 
Go and Uber’s time series database m3
Go and Uber’s time series database m3Go and Uber’s time series database m3
Go and Uber’s time series database m3
 
Time Series Data in a Time Series World
Time Series Data in a Time Series WorldTime Series Data in a Time Series World
Time Series Data in a Time Series World
 
Apache Solr as a compressed, scalable, and high performance time series database
Apache Solr as a compressed, scalable, and high performance time series databaseApache Solr as a compressed, scalable, and high performance time series database
Apache Solr as a compressed, scalable, and high performance time series database
 
Pain points with M3, some things to address them and how replication works
Pain points with M3, some things to address them and how replication worksPain points with M3, some things to address them and how replication works
Pain points with M3, some things to address them and how replication works
 
FOSDEM 2020: Querying over millions and billions of metrics with M3DB's index
FOSDEM 2020: Querying over millions and billions of metrics with M3DB's indexFOSDEM 2020: Querying over millions and billions of metrics with M3DB's index
FOSDEM 2020: Querying over millions and billions of metrics with M3DB's index
 
Advanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXAdvanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMX
 
The new time series kid on the block
The new time series kid on the blockThe new time series kid on the block
The new time series kid on the block
 
Chronix Poster for the Poster Session FAST 2017
Chronix Poster for the Poster Session FAST 2017Chronix Poster for the Poster Session FAST 2017
Chronix Poster for the Poster Session FAST 2017
 
A Fast and Efficient Time Series Storage Based on Apache Solr
A Fast and Efficient Time Series Storage Based on Apache SolrA Fast and Efficient Time Series Storage Based on Apache Solr
A Fast and Efficient Time Series Storage Based on Apache Solr
 
Efficient and Fast Time Series Storage - The missing link in dynamic software...
Efficient and Fast Time Series Storage - The missing link in dynamic software...Efficient and Fast Time Series Storage - The missing link in dynamic software...
Efficient and Fast Time Series Storage - The missing link in dynamic software...
 
Performance evaluation of apache tajo
Performance evaluation of apache tajoPerformance evaluation of apache tajo
Performance evaluation of apache tajo
 
2013 05 ny
2013 05 ny2013 05 ny
2013 05 ny
 
Discretized Stream - Fault-Tolerant Streaming Computation at Scale - SOSP
Discretized Stream - Fault-Tolerant Streaming Computation at Scale - SOSPDiscretized Stream - Fault-Tolerant Streaming Computation at Scale - SOSP
Discretized Stream - Fault-Tolerant Streaming Computation at Scale - SOSP
 
Data Structures and Performance for Scientific Computing with Hadoop and Dumb...
Data Structures and Performance for Scientific Computing with Hadoop and Dumb...Data Structures and Performance for Scientific Computing with Hadoop and Dumb...
Data Structures and Performance for Scientific Computing with Hadoop and Dumb...
 
OSDC 2016 - Chronix - A fast and efficient time series storage based on Apach...
OSDC 2016 - Chronix - A fast and efficient time series storage based on Apach...OSDC 2016 - Chronix - A fast and efficient time series storage based on Apach...
OSDC 2016 - Chronix - A fast and efficient time series storage based on Apach...
 
ToroDB: scaling PostgreSQL like MongoDB / Álvaro Hernández Tortosa (8Kdata)
ToroDB: scaling PostgreSQL like MongoDB / Álvaro Hernández Tortosa (8Kdata)ToroDB: scaling PostgreSQL like MongoDB / Álvaro Hernández Tortosa (8Kdata)
ToroDB: scaling PostgreSQL like MongoDB / Álvaro Hernández Tortosa (8Kdata)
 
Query optimization in Apache Tajo
Query optimization in Apache TajoQuery optimization in Apache Tajo
Query optimization in Apache Tajo
 
Gnocchi v4 (preview)
Gnocchi v4 (preview)Gnocchi v4 (preview)
Gnocchi v4 (preview)
 
RDO hangout on gnocchi
RDO hangout on gnocchiRDO hangout on gnocchi
RDO hangout on gnocchi
 

Destacado

HBaseCon 2015: OpenTSDB and AsyncHBase Update
HBaseCon 2015: OpenTSDB and AsyncHBase UpdateHBaseCon 2015: OpenTSDB and AsyncHBase Update
HBaseCon 2015: OpenTSDB and AsyncHBase UpdateHBaseCon
 
Network Monitoring with Icinga
Network Monitoring with IcingaNetwork Monitoring with Icinga
Network Monitoring with Icingalearjk
 
opentsdb in a real enviroment
opentsdb in a real enviromentopentsdb in a real enviroment
opentsdb in a real enviromentChen Robert
 
Monitoring MySQL with OpenTSDB
Monitoring MySQL with OpenTSDBMonitoring MySQL with OpenTSDB
Monitoring MySQL with OpenTSDBGeoffrey Anderson
 
Update on OpenTSDB and AsyncHBase
Update on OpenTSDB and AsyncHBase Update on OpenTSDB and AsyncHBase
Update on OpenTSDB and AsyncHBase HBaseCon
 
HBaseCon 2013: OpenTSDB at Box
HBaseCon 2013: OpenTSDB at BoxHBaseCon 2013: OpenTSDB at Box
HBaseCon 2013: OpenTSDB at BoxCloudera, Inc.
 
Open TSDB Lightning Talk
Open TSDB Lightning TalkOpen TSDB Lightning Talk
Open TSDB Lightning TalkCloudOps2005
 
Monitoring @ scale over diverse data sources @ PayPal - Druid, TSDB, Hadoop
Monitoring @ scale over diverse data sources @ PayPal  - Druid, TSDB, HadoopMonitoring @ scale over diverse data sources @ PayPal  - Druid, TSDB, Hadoop
Monitoring @ scale over diverse data sources @ PayPal - Druid, TSDB, HadoopSenthil Pandurangan
 
[FR] Timeseries appliqué aux couches de bébé
[FR] Timeseries appliqué aux couches de bébé[FR] Timeseries appliqué aux couches de bébé
[FR] Timeseries appliqué aux couches de bébéOVHcloud
 
Argus Production Monitoring at Salesforce
Argus Production Monitoring at SalesforceArgus Production Monitoring at Salesforce
Argus Production Monitoring at SalesforceHBaseCon
 
Apache Phoenix: Use Cases and New Features
Apache Phoenix: Use Cases and New FeaturesApache Phoenix: Use Cases and New Features
Apache Phoenix: Use Cases and New FeaturesHBaseCon
 

Destacado (14)

HBaseCon 2015: OpenTSDB and AsyncHBase Update
HBaseCon 2015: OpenTSDB and AsyncHBase UpdateHBaseCon 2015: OpenTSDB and AsyncHBase Update
HBaseCon 2015: OpenTSDB and AsyncHBase Update
 
Network Monitoring with Icinga
Network Monitoring with IcingaNetwork Monitoring with Icinga
Network Monitoring with Icinga
 
opentsdb in a real enviroment
opentsdb in a real enviromentopentsdb in a real enviroment
opentsdb in a real enviroment
 
Monitoring MySQL with OpenTSDB
Monitoring MySQL with OpenTSDBMonitoring MySQL with OpenTSDB
Monitoring MySQL with OpenTSDB
 
Update on OpenTSDB and AsyncHBase
Update on OpenTSDB and AsyncHBase Update on OpenTSDB and AsyncHBase
Update on OpenTSDB and AsyncHBase
 
HBaseCon 2013: OpenTSDB at Box
HBaseCon 2013: OpenTSDB at BoxHBaseCon 2013: OpenTSDB at Box
HBaseCon 2013: OpenTSDB at Box
 
Graph everything
Graph everythingGraph everything
Graph everything
 
Open TSDB Lightning Talk
Open TSDB Lightning TalkOpen TSDB Lightning Talk
Open TSDB Lightning Talk
 
Monitoring @ scale over diverse data sources @ PayPal - Druid, TSDB, Hadoop
Monitoring @ scale over diverse data sources @ PayPal  - Druid, TSDB, HadoopMonitoring @ scale over diverse data sources @ PayPal  - Druid, TSDB, Hadoop
Monitoring @ scale over diverse data sources @ PayPal - Druid, TSDB, Hadoop
 
[FR] Timeseries appliqué aux couches de bébé
[FR] Timeseries appliqué aux couches de bébé[FR] Timeseries appliqué aux couches de bébé
[FR] Timeseries appliqué aux couches de bébé
 
Level db
Level dbLevel db
Level db
 
時系列の世界の時系列データ
時系列の世界の時系列データ時系列の世界の時系列データ
時系列の世界の時系列データ
 
Argus Production Monitoring at Salesforce
Argus Production Monitoring at SalesforceArgus Production Monitoring at Salesforce
Argus Production Monitoring at Salesforce
 
Apache Phoenix: Use Cases and New Features
Apache Phoenix: Use Cases and New FeaturesApache Phoenix: Use Cases and New Features
Apache Phoenix: Use Cases and New Features
 

Ähnlich wie openTSDB - Metrics for a distributed world

OSMC 2013 | openTSDB - metrics for a distributed world
OSMC 2013 | openTSDB - metrics for a distributed worldOSMC 2013 | openTSDB - metrics for a distributed world
OSMC 2013 | openTSDB - metrics for a distributed worldNETWAYS
 
Real Time Analytics with Apache Cassandra - Cassandra Day Munich
Real Time Analytics with Apache Cassandra - Cassandra Day MunichReal Time Analytics with Apache Cassandra - Cassandra Day Munich
Real Time Analytics with Apache Cassandra - Cassandra Day MunichGuido Schmutz
 
Kusto (Azure Data Explorer) Training for R&D - January 2019
Kusto (Azure Data Explorer) Training for R&D - January 2019 Kusto (Azure Data Explorer) Training for R&D - January 2019
Kusto (Azure Data Explorer) Training for R&D - January 2019 Tal Bar-Zvi
 
Mapreduce Algorithms
Mapreduce AlgorithmsMapreduce Algorithms
Mapreduce AlgorithmsAmund Tveit
 
Cassandra sf meetup_2013_07_31
Cassandra sf meetup_2013_07_31Cassandra sf meetup_2013_07_31
Cassandra sf meetup_2013_07_31George Courtsunis
 
Cassandra at Disqus — SF Cassandra Users Group July 31st
Cassandra at Disqus — SF Cassandra Users Group July 31stCassandra at Disqus — SF Cassandra Users Group July 31st
Cassandra at Disqus — SF Cassandra Users Group July 31stDataStax Academy
 
Bio-IT Asia 2013: Informatics & Cloud - Best Practices & Lessons Learned
Bio-IT Asia 2013: Informatics & Cloud - Best Practices & Lessons LearnedBio-IT Asia 2013: Informatics & Cloud - Best Practices & Lessons Learned
Bio-IT Asia 2013: Informatics & Cloud - Best Practices & Lessons LearnedChris Dagdigian
 
Multi dimensional profiling
Multi dimensional profilingMulti dimensional profiling
Multi dimensional profilingbergel
 
M|18 Analytics in the Real World, Case Studies and Use Cases
M|18 Analytics in the Real World, Case Studies and Use CasesM|18 Analytics in the Real World, Case Studies and Use Cases
M|18 Analytics in the Real World, Case Studies and Use CasesMariaDB plc
 
Real Time Analytics with Apache Cassandra - Cassandra Day Berlin
Real Time Analytics with Apache Cassandra - Cassandra Day BerlinReal Time Analytics with Apache Cassandra - Cassandra Day Berlin
Real Time Analytics with Apache Cassandra - Cassandra Day BerlinGuido Schmutz
 
Bio-IT & Cloud Sobriety: 2013 Beyond The Genome Meeting
Bio-IT & Cloud Sobriety: 2013 Beyond The Genome MeetingBio-IT & Cloud Sobriety: 2013 Beyond The Genome Meeting
Bio-IT & Cloud Sobriety: 2013 Beyond The Genome MeetingChris Dagdigian
 
Computação Paralela: Benefícios e Desafios - Intel Software Conference 2013
Computação Paralela: Benefícios e Desafios - Intel Software Conference 2013Computação Paralela: Benefícios e Desafios - Intel Software Conference 2013
Computação Paralela: Benefícios e Desafios - Intel Software Conference 2013Intel Software Brasil
 
A Hacking Toolset for Big Tabular Files (3)
A Hacking Toolset for Big Tabular Files (3)A Hacking Toolset for Big Tabular Files (3)
A Hacking Toolset for Big Tabular Files (3)Toshiyuki Shimono
 
M|18 Understanding the Architecture of MariaDB ColumnStore
M|18 Understanding the Architecture of MariaDB ColumnStoreM|18 Understanding the Architecture of MariaDB ColumnStore
M|18 Understanding the Architecture of MariaDB ColumnStoreMariaDB plc
 

Ähnlich wie openTSDB - Metrics for a distributed world (20)

OSMC 2013 | openTSDB - metrics for a distributed world
OSMC 2013 | openTSDB - metrics for a distributed worldOSMC 2013 | openTSDB - metrics for a distributed world
OSMC 2013 | openTSDB - metrics for a distributed world
 
Real Time Analytics with Apache Cassandra - Cassandra Day Munich
Real Time Analytics with Apache Cassandra - Cassandra Day MunichReal Time Analytics with Apache Cassandra - Cassandra Day Munich
Real Time Analytics with Apache Cassandra - Cassandra Day Munich
 
Kusto (Azure Data Explorer) Training for R&D - January 2019
Kusto (Azure Data Explorer) Training for R&D - January 2019 Kusto (Azure Data Explorer) Training for R&D - January 2019
Kusto (Azure Data Explorer) Training for R&D - January 2019
 
Mapreduce Algorithms
Mapreduce AlgorithmsMapreduce Algorithms
Mapreduce Algorithms
 
Cassandra sf meetup_2013_07_31
Cassandra sf meetup_2013_07_31Cassandra sf meetup_2013_07_31
Cassandra sf meetup_2013_07_31
 
Cassandra at Disqus — SF Cassandra Users Group July 31st
Cassandra at Disqus — SF Cassandra Users Group July 31stCassandra at Disqus — SF Cassandra Users Group July 31st
Cassandra at Disqus — SF Cassandra Users Group July 31st
 
Bio-IT Asia 2013: Informatics & Cloud - Best Practices & Lessons Learned
Bio-IT Asia 2013: Informatics & Cloud - Best Practices & Lessons LearnedBio-IT Asia 2013: Informatics & Cloud - Best Practices & Lessons Learned
Bio-IT Asia 2013: Informatics & Cloud - Best Practices & Lessons Learned
 
noSQL @ QCon SP
noSQL @ QCon SPnoSQL @ QCon SP
noSQL @ QCon SP
 
Multi dimensional profiling
Multi dimensional profilingMulti dimensional profiling
Multi dimensional profiling
 
M|18 Analytics in the Real World, Case Studies and Use Cases
M|18 Analytics in the Real World, Case Studies and Use CasesM|18 Analytics in the Real World, Case Studies and Use Cases
M|18 Analytics in the Real World, Case Studies and Use Cases
 
GeeCON Prague 2015
GeeCON Prague 2015GeeCON Prague 2015
GeeCON Prague 2015
 
Real Time Analytics with Apache Cassandra - Cassandra Day Berlin
Real Time Analytics with Apache Cassandra - Cassandra Day BerlinReal Time Analytics with Apache Cassandra - Cassandra Day Berlin
Real Time Analytics with Apache Cassandra - Cassandra Day Berlin
 
Bio-IT & Cloud Sobriety: 2013 Beyond The Genome Meeting
Bio-IT & Cloud Sobriety: 2013 Beyond The Genome MeetingBio-IT & Cloud Sobriety: 2013 Beyond The Genome Meeting
Bio-IT & Cloud Sobriety: 2013 Beyond The Genome Meeting
 
Microsoft Dryad
Microsoft DryadMicrosoft Dryad
Microsoft Dryad
 
The R of War
The R of WarThe R of War
The R of War
 
Computação Paralela: Benefícios e Desafios - Intel Software Conference 2013
Computação Paralela: Benefícios e Desafios - Intel Software Conference 2013Computação Paralela: Benefícios e Desafios - Intel Software Conference 2013
Computação Paralela: Benefícios e Desafios - Intel Software Conference 2013
 
Slide1
Slide1Slide1
Slide1
 
A Hacking Toolset for Big Tabular Files (3)
A Hacking Toolset for Big Tabular Files (3)A Hacking Toolset for Big Tabular Files (3)
A Hacking Toolset for Big Tabular Files (3)
 
M|18 Understanding the Architecture of MariaDB ColumnStore
M|18 Understanding the Architecture of MariaDB ColumnStoreM|18 Understanding the Architecture of MariaDB ColumnStore
M|18 Understanding the Architecture of MariaDB ColumnStore
 
BDACA - Lecture7
BDACA - Lecture7BDACA - Lecture7
BDACA - Lecture7
 

Último

AWS for the beginning is cloud computing
AWS for the beginning  is  cloud computingAWS for the beginning  is  cloud computing
AWS for the beginning is cloud computingkajalghule1
 
My self introduction to know others abut me
My self  introduction to know others abut meMy self  introduction to know others abut me
My self introduction to know others abut meManoj Prabakar B
 
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfLLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfThomas Poetter
 
Put a flag on it. A busy developer's guide to feature toggles.
Put a flag on it. A busy developer's guide to feature toggles.Put a flag on it. A busy developer's guide to feature toggles.
Put a flag on it. A busy developer's guide to feature toggles.Mateusz Kwasniewski
 
Navigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersNavigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersOnePlan Solutions
 
Azure Migration Guide for IT Professionals
Azure Migration Guide for IT ProfessionalsAzure Migration Guide for IT Professionals
Azure Migration Guide for IT ProfessionalsChristine Shepherd
 
Q1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupQ1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupMemory Fabric Forum
 
Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Daniel Toomey
 
My sample product research idea for you!
My sample product research idea for you!My sample product research idea for you!
My sample product research idea for you!KivenRaySarsaba
 
A Comprehensive Theoretical Overview of Self-Driving Car Technology
A Comprehensive Theoretical Overview of Self-Driving Car TechnologyA Comprehensive Theoretical Overview of Self-Driving Car Technology
A Comprehensive Theoretical Overview of Self-Driving Car TechnologyKumar Bipin
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS Chicago
 
Enhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersEnhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersThousandEyes
 
CIRCLE geometry lesson 2nd quarter grade 10
CIRCLE geometry lesson 2nd quarter  grade 10CIRCLE geometry lesson 2nd quarter  grade 10
CIRCLE geometry lesson 2nd quarter grade 10RicFernandez4
 
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfOTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfPaige Cruz
 
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfZ-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfDomotica daVinci
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!XfilesPro
 
Cfgmgmtcamp 2024 — eBPF-based Security Observability & Runtime Enforcement wi...
Cfgmgmtcamp 2024 — eBPF-based Security Observability & Runtime Enforcement wi...Cfgmgmtcamp 2024 — eBPF-based Security Observability & Runtime Enforcement wi...
Cfgmgmtcamp 2024 — eBPF-based Security Observability & Runtime Enforcement wi...Raphaël PINSON
 
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptxssuser796efb
 

Último (20)

Russia Ukraine war Cyberspace operations (2022-2024)
Russia Ukraine war Cyberspace operations (2022-2024)Russia Ukraine war Cyberspace operations (2022-2024)
Russia Ukraine war Cyberspace operations (2022-2024)
 
COE AI Lab Universities
COE AI Lab UniversitiesCOE AI Lab Universities
COE AI Lab Universities
 
AWS for the beginning is cloud computing
AWS for the beginning  is  cloud computingAWS for the beginning  is  cloud computing
AWS for the beginning is cloud computing
 
My self introduction to know others abut me
My self  introduction to know others abut meMy self  introduction to know others abut me
My self introduction to know others abut me
 
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfLLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
 
Put a flag on it. A busy developer's guide to feature toggles.
Put a flag on it. A busy developer's guide to feature toggles.Put a flag on it. A busy developer's guide to feature toggles.
Put a flag on it. A busy developer's guide to feature toggles.
 
Navigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersNavigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio Leaders
 
Azure Migration Guide for IT Professionals
Azure Migration Guide for IT ProfessionalsAzure Migration Guide for IT Professionals
Azure Migration Guide for IT Professionals
 
Q1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupQ1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product Lineup
 
Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024
 
My sample product research idea for you!
My sample product research idea for you!My sample product research idea for you!
My sample product research idea for you!
 
A Comprehensive Theoretical Overview of Self-Driving Car Technology
A Comprehensive Theoretical Overview of Self-Driving Car TechnologyA Comprehensive Theoretical Overview of Self-Driving Car Technology
A Comprehensive Theoretical Overview of Self-Driving Car Technology
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user group
 
Enhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersEnhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for Partners
 
CIRCLE geometry lesson 2nd quarter grade 10
CIRCLE geometry lesson 2nd quarter  grade 10CIRCLE geometry lesson 2nd quarter  grade 10
CIRCLE geometry lesson 2nd quarter grade 10
 
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfOTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
 
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfZ-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
 
Cfgmgmtcamp 2024 — eBPF-based Security Observability & Runtime Enforcement wi...
Cfgmgmtcamp 2024 — eBPF-based Security Observability & Runtime Enforcement wi...Cfgmgmtcamp 2024 — eBPF-based Security Observability & Runtime Enforcement wi...
Cfgmgmtcamp 2024 — eBPF-based Security Observability & Runtime Enforcement wi...
 
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
 

openTSDB - Metrics for a distributed world

  • 1. openTSDB - Metrics for a distributed world Oliver Hankeln / gutefrage.net @mydalon Mittwoch, 30. Oktober 13
  • 2. Who am I? Senior Engineer - Data and Infrastructure at gutefrage.net GmbH Was doing software development before DevOps advocate Mittwoch, 30. Oktober 13
  • 3. Who is Gutefrage.net? Germany‘s biggest Q&A platform #1 German site (mobile) about 5M Unique Users #3 German site (desktop) about 17M Unique Users > 4 Mio PI/day Part of the Holtzbrinck group Running several platforms (Gutefrage.net, Helpster.de, Cosmiq, Comprano, ...) Mittwoch, 30. Oktober 13
  • 4. What you will get Why we chose openTSDB What is openTSDB? How does openTSDB store the data? Our experiences Some advice Mittwoch, 30. Oktober 13
  • 6. We were looking at some options Munin Graphite openTSDB Ganglia Scales well no sort of yes yes Keeps all data no no yes no Creating metrics easy easy easy easy Mittwoch, 30. Oktober 13
  • 7. We have a winner! Graphite openTSDB Scales well no sort of Keeps all data no no Creating metrics easy easy Mittwoch, 30. Oktober 13 Bingo! Munin Ganglia yes yes yes no easy easy
  • 9. Separation of concerns $ unzip|strip|touch|finger|grep|mount|fsck|more|yes| fsck|fsck|fsck|umount|sleep UI was not important for our decision Alerting is not what we are looking for in our time series data base Mittwoch, 30. Oktober 13
  • 10. The ecosystem App feeds metrics in via RabbitMQ We base Icinga checks on the metrics We evaluate Skyline and Oculus by Etsy for anomaly detection We deploy sensors via chef Mittwoch, 30. Oktober 13
  • 11. openTSDB Written by Benoît Sigoure at StumbleUpon OpenSource (get it from github) Uses HBase (which is based on HDFS) as a storage Distributed system (multiple TSDs) Mittwoch, 30. Oktober 13
  • 12. The big picture UI tcollector API Mittwoch, 30. Oktober 13 TSD TSD TSD TSD This is really a cluster HBase
  • 13. Putting data into openTSDB $ telnet tsd01.acme.com 4242 put proc.load.avg5min 1382536472 23.2 host=db01.acme.com Mittwoch, 30. Oktober 13
  • 14. It gets even better tcollector is a python script that runs your collectors handles network connection, starts your collectors at set intervals does basic process management adds host tag, does deduplication Mittwoch, 30. Oktober 13
  • 15. A simple tcollector script #!/usr/bin/php <?php #Cast a die $die = rand(1,6); echo "roll.a.d6 " . time() . " " . $die . "n"; Mittwoch, 30. Oktober 13
  • 16. What was that HDFS again? HDFS is a distributed filesystem suitable for Petabytes of data on thousands of machines. Runs on commodity hardware Takes care of redundancy Used by e.g. Facebook, Spotify, eBay,... Mittwoch, 30. Oktober 13
  • 17. Okay... and HBase? HBase is a NoSQL database / data store on top of HDFS Modeled after Google‘s BigTable Built for big tables (billions of rows, millions of columns) Automatic sharding by row key Mittwoch, 30. Oktober 13
  • 18. How openTSDB stores the data Mittwoch, 30. Oktober 13
  • 19. Keys are key! Data is sharded across regions based on their row key You query data based on the row key You can query row key ranges (say e.g. A...D) So: think about key design Mittwoch, 30. Oktober 13
  • 20. Take 1 Row key format: timestamp, metric id Mittwoch, 30. Oktober 13
  • 21. Take 1 Row key format: timestamp, metric id 1382536472, 5 17 Server A Server B Mittwoch, 30. Oktober 13
  • 22. Take 1 Row key format: timestamp, metric id 1382536472, 5 1382536472, 6 17 24 Server A Server B Mittwoch, 30. Oktober 13
  • 23. Take 1 Row key format: timestamp, metric id 1382536472, 5 1382536472, 6 1382536472, 8 1382536473, 5 1382536473, 6 1382536473, 8 Mittwoch, 30. Oktober 13 17 24 12 134 10 99 Server A Server B
  • 24. Take 1 Row key format: timestamp, metric id 1382536472, 5 1382536472, 6 1382536472, 8 1382536473, 5 1382536473, 6 1382536473, 8 1382536474, 5 1382536474, 6 Mittwoch, 30. Oktober 13 17 24 12 134 10 99 12 42 Server A Server B
  • 25. Solution: Swap timestamp and metric id Row key format: metric id, timestamp 5, 1382536472 6, 1382536472 8, 1382536472 5, 1382536473 6, 1382536473 8, 1382536473 5, 1382536474 6, 1382536474 Mittwoch, 30. Oktober 13 17 24 12 134 10 99 12 42 Server A Server B
  • 26. Solution: Swap timestamp and metric id Row key format: metric id, timestamp 5, 1382536472 6, 1382536472 8, 1382536472 5, 1382536473 6, 1382536473 8, 1382536473 5, 1382536474 6, 1382536474 Mittwoch, 30. Oktober 13 17 24 12 134 10 99 12 42 Server A Server B
  • 27. Take 2 Metric ID first, then timestamp Searching through many rows is slower than searching through viewer rows. (Obviously) So: Put multiple data points into one row Mittwoch, 30. Oktober 13
  • 28. Take 2 continued 5, 1382608800 5, 1382612400 Mittwoch, 30. Oktober 13 +23 +35 +94 +142 17 1 23 42 +13 +25 +88 +89 3 44 12 2
  • 29. Take 2 continued Row key 5, 1382608800 5, 1382612400 Mittwoch, 30. Oktober 13 +23 +35 +94 +142 17 1 23 42 +13 +25 +88 +89 3 44 12 2
  • 30. Take 2 continued Cell Name Row key 5, 1382608800 5, 1382612400 Mittwoch, 30. Oktober 13 +23 +35 +94 +142 17 1 23 42 +13 +25 +88 +89 3 44 12 2
  • 31. Take 2 continued Cell Name Row key 5, 1382608800 5, 1382612400 Mittwoch, 30. Oktober 13 Data point +23 +35 +94 +142 17 1 23 42 +13 +25 +88 +89 3 44 12 2
  • 32. Where are the tags stored? They are put at the end of the row key Both tag names and tag values are represented by IDs Mittwoch, 30. Oktober 13
  • 33. The Row Key 3 Bytes - metric ID 4 Bytes - timestamp (rounded down to the hour) 3 Bytes tag ID 3 Bytes tag value ID Total: 7 Bytes + 6 Bytes * Number of tags Mittwoch, 30. Oktober 13
  • 34. Let‘s look at some graphs Mittwoch, 30. Oktober 13
  • 36. Myth: Keeping Data is expensive Gartner found the price for enterprise SSDs at 1$/GB in 2013 A data point gets compressed to 2-3 Bytes A metric that you measure every second then uses disk space for 18.9ct per year. Usually it is even cheaper Mittwoch, 30. Oktober 13
  • 37. If your work costs 50$ per hour and it takes you only one minute to think about and configure your RRD compaction setting, you could have collected that metric on a second-by-second basis for 4.4 YEARS instead. Mittwoch, 30. Oktober 13
  • 38. Myth: the amount of metrics is too limited Don‘t confuse Graphite metric count with openTSBD metric count. 3 Bytes of metric ID = 16.7M possibilities 3 Bytes tag value ID = 16.7M possibilities => at least 280 T metrics (graphite counting) Mittwoch, 30. Oktober 13
  • 40. Tools shape culture shapes tools It is time for a new monitoring culture! Embrace machine learning! Monitor everything in your organisation! Throw of the shackles of fixed intervals! Come, join the revolution! Mittwoch, 30. Oktober 13
  • 42. What works well We store about 200M data points in several thousand time series with no issues tcollector is decoupling measurement from storage Creating new metrics is really easy You are free to choose your rhythm Mittwoch, 30. Oktober 13
  • 43. Challenges The UI is seriously lacking no annotation support out of the box no meta data for time series Only 1s time resolution (and only 1 value/s/ time series) Mittwoch, 30. Oktober 13
  • 44. salvation is coming OpenTSDB 2 is around the corner millisecond precision annotations and meta data improved API improved UI Mittwoch, 30. Oktober 13
  • 45. Friendly advice Pick a naming scheme and stick to it Use tags wisely (not more than 6 or 7 tags per data point) Use tcollector wait for openTSDB 2 ;-) Mittwoch, 30. Oktober 13
  • 46. Questions? Please contact me: oliver.hankeln@gutefrage.net @mydalon I‘ll upload the slides and tweet about it Mittwoch, 30. Oktober 13