4. 4
web server
web server
web server
EC2で生成 S3に集約 EMRで集計
FluentdS3Plugin
S3DistCp
hiveorstreaming
RDSに出力
/main?page=1
/main?page=2
/main?page=1
{“page”:”1”}
{“page”:”2”}
{“page”:”1”}
page1 t 2
page2 t 1
page | count
---------------
page1 | 2
page2 | 1
7. 7
web server
web server
web server
EC2で生成 S3に集約 EMRで集計
FluentdS3Plugin
S3DistCp
hiveorstreaming
RDSに出力
/main?page=1
/main?page=2
/main?page=1
{“page”:”1”}
{“page”:”2”}
{“page”:”1”}
page1 t 2
page2 t 1
page | count
---------------
page1 | 2
page2 | 1
完成までは試行錯誤と場当たり処理の繰り返
し
21. 200 vs それ以外
| parse regex "t(?<status>ddd?)th"
| timeslice 1m
| if(status = "200", 1,0) as statGood
| if(status <> "200", 1,0) as statOther
| sum(statGood) as good, sum(statOther) as bad by _timeslice
| sort by + _timeslice
22. 22
web server
web server
web server
EC2で生成 S3に集約 EMRで集計
FluentdS3Plugin
S3DistCp
hiveorstreaming
RDSに出力
/main?page=1
/main?page=2
/main?page=1
{“page”:”1”}
{“page”:”2”}
{“page”:”1”}
page1 t 2
page2 t 1
page | count
---------------
page1 | 2
page2 | 1
ログ収集、解析
をあきらめない
完成までは試行錯誤と場当たり処理の繰り返
し