Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Stream processing in Mercari - Devsumi 2015 autumn LT

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Nächste SlideShare
KOWAZA for mackerel
KOWAZA for mackerel
Wird geladen in …3
×

Hier ansehen

1 von 20 Anzeige
Anzeige

Weitere Verwandte Inhalte

Diashows für Sie (20)

Andere mochten auch (20)

Anzeige

Ähnlich wie Stream processing in Mercari - Devsumi 2015 autumn LT (20)

Weitere von Masahiro Nagano (20)

Anzeige

Aktuellste (20)

Stream processing in Mercari - Devsumi 2015 autumn LT

  1. 1. Stream processing in Mercari Developers Summit 2015 Autumn LT @kazeburo Masahiro Nagano
  2. 2. Me •長野雅広(Masahiro Nagano) •@kazeburo •Mercari, Inc. •Principal Engineer, Site Reliability
  3. 3. Your Friendly Mobile Marketplace
  4. 4. Stream processing for Monitoring
  5. 5. Monitoring
  6. 6. collect metrics value • Just get current value/point • e.g. load avg, memory usage • Get counter/incremental value and calculate changes • e.g. traffic, cpu usage • Parse log and count value by your self • e.g. access_log, error_log, custom_log
  7. 7. Log monitoring Past and present
  8. 8. While ago • Make a script for parsing and count logs • Execute periodically from monitoring tool • Too many tail (io) and grep (cpu) (;_;) #!/bin/sh set -e LOG_WATCH=20000 TARGET=$(date -d '61 second ago' +%H:%M:[0-9][0-9]) tail -$LOG_WATCH /var/log/httpd/service_access_log | grep "$TARGET" | wc -l tail -$LOG_WATCH /var/log/httpd/service_access_log | grep "$TARGET" | grep '" 500 ' wc -l
  9. 9. fluentd + datacounter Web access_log in_tail Web access_log Web access_log aggregate datacounter <match mercari.access_log> type datacounter count_interval 1m count_key status pattern1 2xx ^2dd$ pattern2 3xx ^3dd$ pattern4 4xx ^4dd$ pattern5 5xx ^5dd$ tag status.datacount.web </match> <match status.datacount.web> type zabbix zabbix_server 10.x.x.x </match>
  10. 10. Cons of fluentd +datacounter •Gigantic fluend.conf •need a configuration file generator? •Write fluend plugin for responding various needs? •restart fluentd is required for adding a new tabulation
  11. 11. Norikra •Norikra is an awesome open source product by Mr. tagomori, provides “Stream processing” with SQL •We LOVE SQL!!
  12. 12. 219.109.xx.xx - - [09/Nov/2014:06:50:07 +0900] "GET /mt.js HTTP/1.1" 200 6339 "http://blog.nomadscafe.jp/2013/06/webdb-pressvol172.html" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" 50.116.xx.xx - - [09/Nov/2014:06:54:41 +0900] "GET /2014/09/line- isucon4-51192.html HTTP/1.1" 200 30194 "http://www.google.co.uk/url? sa=t&source=web&cd=1" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.92 Safari/535.2" 76.164.xx.xx - - [09/Nov/2014:07:09:21 +0900] "GET / HTTP/1.0" 200 75606 "http://blog.nomadscafe.jp/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/ 537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36" SELECT COUNT(1, status like “5%”) AS count_5x, COUNT(1, status like “2%”) AS count_2x FROM access_log.win:time_batch(1 min) { “count_2x”: 765, “count_5x”: 2 } Norikra
  13. 13. After Norika Web access_log in_tail Web access_log Web access_log aggregate SQL投入 Norikra
  14. 14. Pros of Norikra • Do not need a gigantic configuration file • Do not required restarting a daemon • Schema less • analysis any log data like error_log, application specific log • SQL • every engineer/producer can write SQL
  15. 15. Graph/Alert •zabbix •nagios •GrowthForecast (graph only) •Data Dog •mackerel
  16. 16. Worker/Batch WebWeb Stream Processing in Mercari Web/App access_log Worker/Batch create SQL Norikra error_log app_log error_log app_log define Alert Mackerel Generates Graph automatically!!
  17. 17. Alternatives and Competitors • PipelineDB • www.pipelinedb.com • Azure Stream Analytics • azure.microsoft.com/ja-jp/services/stream- analytics/ • Amazon Kinesis Analytics(coming soon) • aws.amazon.com/kinesis/analytics/
  18. 18. end

×