Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
1
Ted Jung
Consulting Architect
Observable Kubernetes
with Elastic Stack
2
Ted, Jung
Consulting Architect @Elastic
3
• Intro to Elastic Stack
• Kubernetes (challenges, visibility)
• Logging and Monitoring Kubernetes
• APM tracing
• Demo
...
44
Intro to Elastic Stack
● Elastic stack
● Elastic Elasticsearch Service
5
Elastic Stack
Store, Search, & AnalyzeElasticsearch
Visualize & ManageKibana
Beats IngestLogstash
SaaS
Elasticsearch
Ser...
6
US West
(Oregon)
US West
(N. Cal)
US East (N.
Virginia)
EU
(Ireland)
EU
(Germany)
S. America
(East)
Asia Pacific
(Sydney...
77
Kubernetes
● Challenges
● Visibility
8
Kinds of Kubernetes
9
Container
Runtime
Kubernetes is…
§ Kubernetes is complicated
§ Kubernetes is extremely dynamic, which makes monitoring &...
10
Kubernetes Visibility Challenges…
11
Three Pillars of Observability
Elastic brings a holistic solution
for Kubernetes observability in one platform.
1212
Logging
● Cluster level logging
● Service logging (eg, nginx, mysql)
● Custom application logging
13
• Need for a logging solution
– Kubernetes does not have a native solution
– kubectl logs is too hard for large cluster...
14
kubectl create -f filebeat-kubernetes.yaml
Cluster
Service
Application
Enriched Logging & Better Log Collection
with Fi...
15
filebeat.autodiscover:
providers:
- type: kubernetes
templates:
- condition:
contains:
kubernetes.container.image: "ngi...
16
• Kubernetes module
• Monitors pods and services
– Cluster, pod & container metrics
– Application metrics through auto-...
17
Metricbeat Modules
Simplify collection and visualization of common metrics
● Aerospike module
● Apache module
● Ceph mo...
18
Filebeat Modules
Simplify collection and visualization of common metrics
19
Beats Family
2020
Tracing
● Elastic APM
21
Application Monitoring with Elastic APM
Adding end-user experience and application-level monitoring to the stack
RUM
2222
Observable Kubernetes
Elastic Stack:
Three Pillars of Observability in One Platform
● Logging
● Metrics
● APM Tracing
2323
Demo
24
filebeat
metricbeat
packetbeat
heartbeat
Visualization
indexing
Demo Env
Store
25
Master&WorkerNode 1
Metricbeat
Filebeat
Worker Node 2
Metricbeat
Filebeat
Worker Node 3
Metricbeat
Filebeat
Heartbeat, ...
26
Simple deployment templates to gain insights faster
One-click upgrades of the Elastic Stack
Centrally manage all your d...
27
Howto deploy
Create deployment
Cloud(AWS)
Region
Demo Env
28
Demo Env
29
Access URL: http://**.**.**.**:?????/
Demo Application
30
Nächste SlideShare
Wird geladen in …5
×

Monitoring Kubernetes with Elasticsearch Services - Ted Jung, Consulting Architect, Elastic :: AWS Summit Seoul 2019

630 Aufrufe

Veröffentlicht am

스폰서 발표 세션 | Monitoring Kubernetes with Elasticsearch Services
Ted Jung, Consulting Architect, Elastic

How you can use Elastic Stack products e.g. Elasticsearch, Beats etc to monitor containers running in Kubernetes.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

Monitoring Kubernetes with Elasticsearch Services - Ted Jung, Consulting Architect, Elastic :: AWS Summit Seoul 2019

  1. 1. 1 Ted Jung Consulting Architect Observable Kubernetes with Elastic Stack
  2. 2. 2 Ted, Jung Consulting Architect @Elastic
  3. 3. 3 • Intro to Elastic Stack • Kubernetes (challenges, visibility) • Logging and Monitoring Kubernetes • APM tracing • Demo Agenda
  4. 4. 44 Intro to Elastic Stack ● Elastic stack ● Elastic Elasticsearch Service
  5. 5. 5 Elastic Stack Store, Search, & AnalyzeElasticsearch Visualize & ManageKibana Beats IngestLogstash SaaS Elasticsearch Service Self Managed Elastic Cloud Enterprise Standalone Deployment The Elastic Stack
  6. 6. 6 US West (Oregon) US West (N. Cal) US East (N. Virginia) EU (Ireland) EU (Germany) S. America (East) Asia Pacific (Sydney) Asia Pacific (Tokyo) Asia Pacific (Singapore) Elasticsearch The Official Hosted Elasticsearch & Kibana Solution AWS 9Regions VM or On-Prem VM or On-Prem The Self Managed Elasticsearch & Kibana Solution
  7. 7. 77 Kubernetes ● Challenges ● Visibility
  8. 8. 8 Kinds of Kubernetes
  9. 9. 9 Container Runtime Kubernetes is… § Kubernetes is complicated § Kubernetes is extremely dynamic, which makes monitoring & logging very challenging § All components provide metrics § On top of all this there are a ton of different providers
  10. 10. 10 Kubernetes Visibility Challenges…
  11. 11. 11 Three Pillars of Observability Elastic brings a holistic solution for Kubernetes observability in one platform.
  12. 12. 1212 Logging ● Cluster level logging ● Service logging (eg, nginx, mysql) ● Custom application logging
  13. 13. 13 • Need for a logging solution – Kubernetes does not have a native solution – kubectl logs is too hard for large clusters • Cluster-level logging – Logs have separate storage and lifecycle independent of nodes, pods and containers – Kubernetes provides no native storage solution for log data • Application-level logging – Complicated – Packaged applications (eg. nginx) – Custom applications Kubernetes Logging
  14. 14. 14 kubectl create -f filebeat-kubernetes.yaml Cluster Service Application Enriched Logging & Better Log Collection with Filebeat • Cluster level logging • Service level logs (eg. Nginx logs) – Auto-discovery • Application logs – Cluster level logging (stdout, stderr) – Custom log file
  15. 15. 15 filebeat.autodiscover: providers: - type: kubernetes templates: - condition: contains: kubernetes.container.image: "nginx" config: - module: nginx access: # For nginx access log prospector: type: docker containers.ids: - "${data.kubernetes.container.id}" Filebeat Auto-Discovery
  16. 16. 16 • Kubernetes module • Monitors pods and services – Cluster, pod & container metrics – Application metrics through auto-discovery (eg. Nginx) • Metrics sources - Cover them ALL – Kubelet (heapster, cAdvisor) – kube-state-metric – Kubernetes events – Prometheus module (beta) • Curated infra UI – Dedicated Kibana app Comprehensive Metrics Collection with Metricbeat
  17. 17. 17 Metricbeat Modules Simplify collection and visualization of common metrics ● Aerospike module ● Apache module ● Ceph module ● Couchbase module ● Docker module ● Dropwizard module ● Elasticsearch module ● Etcd module ● Golang module ● Graphite module ● HAProxy module ● HTTP module ● Jolokia module ● Kafka module ● Kibana module ● Kubernetes module ● kvm module ● Logstash module ● Memcached module ● MongoDB module ● Munin module ● MySQL module ● Nginx module ● PHP_FPM module ● PostgreSQL module ● Prometheus module ● RabbitMQ module ● Redis module ● System module ● uwsgi module ● vSphere module ● Windows module ● ZooKeeper module
  18. 18. 18 Filebeat Modules Simplify collection and visualization of common metrics
  19. 19. 19 Beats Family
  20. 20. 2020 Tracing ● Elastic APM
  21. 21. 21 Application Monitoring with Elastic APM Adding end-user experience and application-level monitoring to the stack RUM
  22. 22. 2222 Observable Kubernetes Elastic Stack: Three Pillars of Observability in One Platform ● Logging ● Metrics ● APM Tracing
  23. 23. 2323 Demo
  24. 24. 24 filebeat metricbeat packetbeat heartbeat Visualization indexing Demo Env Store
  25. 25. 25 Master&WorkerNode 1 Metricbeat Filebeat Worker Node 2 Metricbeat Filebeat Worker Node 3 Metricbeat Filebeat Heartbeat, Packetbeat Kubernetes Daemonset for Beat
  26. 26. 26 Simple deployment templates to gain insights faster One-click upgrades of the Elastic Stack Centrally manage all your deployments Enhanced security and performance for all deployments Elastic Elasticsearch Service
  27. 27. 27 Howto deploy Create deployment Cloud(AWS) Region Demo Env
  28. 28. 28 Demo Env
  29. 29. 29 Access URL: http://**.**.**.**:?????/ Demo Application
  30. 30. 30

×