SlideShare ist ein Scribd-Unternehmen logo
1 von 12
Downloaden Sie, um offline zu lesen
OpenTelemetry for Logging
Philipp Krenn, @xeraa
OpenTelemetry
https://github.com/cncf/tag-observability/blob/whitepaper-v1.0.0/whitepaper.md
Create a unified data path for routing traces,
metrics, and logs from apps to backends
Unified
Signal
Routing
Logs have been around forever.
What new value can OpenTelemetry bring?
Easily include common metadata (including
baggage from REST or RPC) in traces, metrics, and
logs to enable correlation
Common
Metadata
Host
Signals and Routing
OTel Collector
Tracing VM
Metrics
App
OTel APM Library
Log Lib
Log Files
Custom
Metrics
Logs
Logs Infra
Metrics
Host Stats K8s API
OTel “hooks”
standard log
frameworks
OTel Collector can
scrape log files and
infra stats in a
vendor-agnostic way
JVM,node,.NET
OTLP to Native OpenTelemetry-compatible backend
Correlation
Decomposing Logs
[2011-10-05T14:48:00] [DEBUG] : Processing Record
[2011-10-05T14:48:00] [INFO] : records_processed=25
[2011-10-05T14:48:00] [WARN] : Record malformatted!
[2011-10-05T14:48:00] [FATAL] :
Exception in thread "main" java.lang.RuntimeException: A test exception
at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13)
at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9)
at com.stackify.StackTraceExample.main(StackTraceExample.java:5)
Decomposing Logs
[2011-10-05T14:48:00] [DEBUG] : Processing Record
[2011-10-05T14:48:00] [INFO] : records_processed=25
[2011-10-05T14:48:00] [WARN] : Record malformatted!
[2011-10-05T14:48:00] [FATAL] :
Exception in thread "main" java.lang.RuntimeException: A test exception
at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13)
at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9)
at com.stackify.StackTraceExample.main(StackTraceExample.java:5)
Record an OpenTelemetry Span
Added Value:
● Latency and success / failure analytics
● Distributed tracing
● Correlate with Business Analytics
● Verbosity is controlled through intelligent sampling
instead of static log-level
Decomposing Logs
[2011-10-05T14:48:00] [DEBUG] : Processing Record
[2011-10-05T14:48:00] [INFO] : records_processed=25
[2011-10-05T14:48:00] [WARN] : Record malformatted!
[2011-10-05T14:48:00] [FATAL] :
Exception in thread "main" java.lang.RuntimeException: A test exception
at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13)
at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9)
at com.stackify.StackTraceExample.main(StackTraceExample.java:5)
Record an OpenTelemetry Metric
Added Value:
● Metric rollups
● Much lower storage costs
● Easily visualized
● Derive Business Intelligence
● Framed in context of transaction
Decomposing Logs
[2011-10-05T14:48:00] [DEBUG] : Processing Record
[2011-10-05T14:48:00] [INFO] : records_processed=25
[2011-10-05T14:48:00] [WARN] : Record malformatted!
[2011-10-05T14:48:00] [FATAL] :
Exception in thread "main" java.lang.RuntimeException: A test exception
at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13)
at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9)
at com.stackify.StackTraceExample.main(StackTraceExample.java:5)
Record an OTel Span Event (the new “log line”)
Added Value:
● Framed in context of transaction
Decomposing Logs
[2011-10-05T14:48:00] [DEBUG] : Processing Record
[2011-10-05T14:48:00] [INFO] : records_processed=25
[2011-10-05T14:48:00] [WARN] : Record malformatted!
[2011-10-05T14:48:00] [FATAL] :
Exception in thread "main" java.lang.RuntimeException: A test exception
at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13)
at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9)
at com.stackify.StackTraceExample.main(StackTraceExample.java:5)
Record an OTel Span Exception
Added Value:
● Derive quality metrics
● Framed in context of transaction
Logs
(OTLP)
File to Elastic Agent
Log Models
Host Collector
Tracing
(OTLP)
App
Instrumentation Lib
Log Lib (eg Logback)
Log File
Metrics
(OTLP)
Logs
(OTLP)
OTel Hook
log(“hello!”)
Hook adds
traceID and
common
metadata
Filebeat
Logs
(Elastic)
OTel logging leverages
existing logging
frameworks
Supported Models: 1
2
3
most mature
1
2
3
least mature
2
3
Direct OTLP
File to Collector OTLP
OpenTelemetry for Logging
Philipp Krenn, @xeraa

Weitere ähnliche Inhalte

Ähnlich wie OSMC 2023 | OpenTelemetry for Logging by Philipp Krenn

Oracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionOracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionAditya Trivedi
 
Combining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityCombining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityElasticsearch
 
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LMESet your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LMEconfluent
 
Jboss World 2011 Infinispan
Jboss World 2011 InfinispanJboss World 2011 Infinispan
Jboss World 2011 Infinispancbo_
 
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Miguel Pérez Colino
 
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...NETWAYS
 
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...James Anderson
 
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...Alex Rayón Jerez
 
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaCombinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaElasticsearch
 
Lap around .net 4
Lap around .net 4Lap around .net 4
Lap around .net 4Abdul Khan
 
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...Zalando Technology
 
Combinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaCombinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaElasticsearch
 
[ACNA2022] Hadoop Vectored IO_ your data just got faster!.pdf
[ACNA2022] Hadoop Vectored IO_ your data just got faster!.pdf[ACNA2022] Hadoop Vectored IO_ your data just got faster!.pdf
[ACNA2022] Hadoop Vectored IO_ your data just got faster!.pdfMukundThakur22
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to KubernetesSamuel Dratwa
 
Aioug ha day oct2015 goldengate- High Availability Day 2015
Aioug ha day oct2015 goldengate- High Availability Day 2015Aioug ha day oct2015 goldengate- High Availability Day 2015
Aioug ha day oct2015 goldengate- High Availability Day 2015aioughydchapter
 

Ähnlich wie OSMC 2023 | OpenTelemetry for Logging by Philipp Krenn (20)

Oracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionOracle 11g data warehouse introdution
Oracle 11g data warehouse introdution
 
Combining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityCombining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observability
 
Oracle Management Cloud
Oracle Management Cloud Oracle Management Cloud
Oracle Management Cloud
 
Oracle Management Cloud
Oracle Management CloudOracle Management Cloud
Oracle Management Cloud
 
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LMESet your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
 
Jboss World 2011 Infinispan
Jboss World 2011 InfinispanJboss World 2011 Infinispan
Jboss World 2011 Infinispan
 
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
 
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
 
Oracle
OracleOracle
Oracle
 
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
 
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
 
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaCombinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
 
Lap around .net 4
Lap around .net 4Lap around .net 4
Lap around .net 4
 
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
 
Combinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaCombinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificada
 
[ACNA2022] Hadoop Vectored IO_ your data just got faster!.pdf
[ACNA2022] Hadoop Vectored IO_ your data just got faster!.pdf[ACNA2022] Hadoop Vectored IO_ your data just got faster!.pdf
[ACNA2022] Hadoop Vectored IO_ your data just got faster!.pdf
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
 
Aioug ha day oct2015 goldengate- High Availability Day 2015
Aioug ha day oct2015 goldengate- High Availability Day 2015Aioug ha day oct2015 goldengate- High Availability Day 2015
Aioug ha day oct2015 goldengate- High Availability Day 2015
 
11g R2
11g R211g R2
11g R2
 
AKS: k8s e azure
AKS: k8s e azureAKS: k8s e azure
AKS: k8s e azure
 

Kürzlich hochgeladen

OC Streetcar Final Presentation-Downtown Santa Ana
OC Streetcar Final Presentation-Downtown Santa AnaOC Streetcar Final Presentation-Downtown Santa Ana
OC Streetcar Final Presentation-Downtown Santa AnaRahsaan L. Browne
 
ServiceNow CIS-Discovery Exam Dumps 2024
ServiceNow CIS-Discovery Exam Dumps 2024ServiceNow CIS-Discovery Exam Dumps 2024
ServiceNow CIS-Discovery Exam Dumps 2024SkillCertProExams
 
DAY 0 8 A Revelation 05-19-2024 PPT.pptx
DAY 0 8 A Revelation 05-19-2024 PPT.pptxDAY 0 8 A Revelation 05-19-2024 PPT.pptx
DAY 0 8 A Revelation 05-19-2024 PPT.pptxFamilyWorshipCenterD
 
art integrated project of computer applications
art integrated project of computer applicationsart integrated project of computer applications
art integrated project of computer applicationsmarvelpwian65
 
Deciding The Topic of our Magazine.pptx.
Deciding The Topic of our Magazine.pptx.Deciding The Topic of our Magazine.pptx.
Deciding The Topic of our Magazine.pptx.bazilnaeem7
 
Breathing in New Life_ Part 3 05 22 2024.pptx
Breathing in New Life_ Part 3 05 22 2024.pptxBreathing in New Life_ Part 3 05 22 2024.pptx
Breathing in New Life_ Part 3 05 22 2024.pptxFamilyWorshipCenterD
 
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docxThe Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docxMogul Press
 
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdf
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdfOracle Database Administration I (1Z0-082) Exam Dumps 2024.pdf
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdfSkillCertProExams
 
Cymulate (Breach and Attack Simulation).
Cymulate (Breach and Attack Simulation).Cymulate (Breach and Attack Simulation).
Cymulate (Breach and Attack Simulation).luckyk1575
 
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdfACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdfKinben Innovation Private Limited
 
05232024 Joint Meeting - Community Networking
05232024 Joint Meeting - Community Networking05232024 Joint Meeting - Community Networking
05232024 Joint Meeting - Community NetworkingMichael Orias
 
Understanding Poverty: A Community Questionnaire
Understanding Poverty: A Community QuestionnaireUnderstanding Poverty: A Community Questionnaire
Understanding Poverty: A Community Questionnairebazilnaeem7
 

Kürzlich hochgeladen (12)

OC Streetcar Final Presentation-Downtown Santa Ana
OC Streetcar Final Presentation-Downtown Santa AnaOC Streetcar Final Presentation-Downtown Santa Ana
OC Streetcar Final Presentation-Downtown Santa Ana
 
ServiceNow CIS-Discovery Exam Dumps 2024
ServiceNow CIS-Discovery Exam Dumps 2024ServiceNow CIS-Discovery Exam Dumps 2024
ServiceNow CIS-Discovery Exam Dumps 2024
 
DAY 0 8 A Revelation 05-19-2024 PPT.pptx
DAY 0 8 A Revelation 05-19-2024 PPT.pptxDAY 0 8 A Revelation 05-19-2024 PPT.pptx
DAY 0 8 A Revelation 05-19-2024 PPT.pptx
 
art integrated project of computer applications
art integrated project of computer applicationsart integrated project of computer applications
art integrated project of computer applications
 
Deciding The Topic of our Magazine.pptx.
Deciding The Topic of our Magazine.pptx.Deciding The Topic of our Magazine.pptx.
Deciding The Topic of our Magazine.pptx.
 
Breathing in New Life_ Part 3 05 22 2024.pptx
Breathing in New Life_ Part 3 05 22 2024.pptxBreathing in New Life_ Part 3 05 22 2024.pptx
Breathing in New Life_ Part 3 05 22 2024.pptx
 
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docxThe Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
 
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdf
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdfOracle Database Administration I (1Z0-082) Exam Dumps 2024.pdf
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdf
 
Cymulate (Breach and Attack Simulation).
Cymulate (Breach and Attack Simulation).Cymulate (Breach and Attack Simulation).
Cymulate (Breach and Attack Simulation).
 
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdfACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
 
05232024 Joint Meeting - Community Networking
05232024 Joint Meeting - Community Networking05232024 Joint Meeting - Community Networking
05232024 Joint Meeting - Community Networking
 
Understanding Poverty: A Community Questionnaire
Understanding Poverty: A Community QuestionnaireUnderstanding Poverty: A Community Questionnaire
Understanding Poverty: A Community Questionnaire
 

OSMC 2023 | OpenTelemetry for Logging by Philipp Krenn

  • 3. Create a unified data path for routing traces, metrics, and logs from apps to backends Unified Signal Routing Logs have been around forever. What new value can OpenTelemetry bring? Easily include common metadata (including baggage from REST or RPC) in traces, metrics, and logs to enable correlation Common Metadata
  • 4. Host Signals and Routing OTel Collector Tracing VM Metrics App OTel APM Library Log Lib Log Files Custom Metrics Logs Logs Infra Metrics Host Stats K8s API OTel “hooks” standard log frameworks OTel Collector can scrape log files and infra stats in a vendor-agnostic way JVM,node,.NET OTLP to Native OpenTelemetry-compatible backend
  • 6. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5)
  • 7. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5) Record an OpenTelemetry Span Added Value: ● Latency and success / failure analytics ● Distributed tracing ● Correlate with Business Analytics ● Verbosity is controlled through intelligent sampling instead of static log-level
  • 8. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5) Record an OpenTelemetry Metric Added Value: ● Metric rollups ● Much lower storage costs ● Easily visualized ● Derive Business Intelligence ● Framed in context of transaction
  • 9. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5) Record an OTel Span Event (the new “log line”) Added Value: ● Framed in context of transaction
  • 10. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5) Record an OTel Span Exception Added Value: ● Derive quality metrics ● Framed in context of transaction
  • 11. Logs (OTLP) File to Elastic Agent Log Models Host Collector Tracing (OTLP) App Instrumentation Lib Log Lib (eg Logback) Log File Metrics (OTLP) Logs (OTLP) OTel Hook log(“hello!”) Hook adds traceID and common metadata Filebeat Logs (Elastic) OTel logging leverages existing logging frameworks Supported Models: 1 2 3 most mature 1 2 3 least mature 2 3 Direct OTLP File to Collector OTLP