2017년 8월 30일 열린 Ad-Tech on AWS 세미나에서 Shinohara Eiji AWS Japan 솔루션즈 아키텍트가 발표한 [Accelerating Ad-Tech on AWS in Japan] 강연의 발표자료입니다.

1. Accelerating AdTech on AWS in Japan
Pragmatic use-cases
Dynalyst / fluct / IM-DMP / UNICORN
Eiji Shinohara ＠shinodogg
Amazon Web Services Japan Solutions Architect

2. 안녕하십니까!
@werner: Amazon CTO
Piljoong
Park-san
Eiji Shinohara (篠原 英治)
 Twitter: @shinodogg
 Blog: shinodogg.com
AWS Solutions Architect
 Market: AdTech & Startup
 Area of Depth: Search
Korean Cuisine Lover
 三겹살/삼겹살, 불고기, 김치,,,

3. Agenda
Japanese AdTech Industry
Japanese AdTech Community
AdTech on AWS use-cases in Japan
Dynalyst http://www.dynalyst.io
fluct https://fluct.jp
IM-DMP https://corp.intimatemerger.com
UNICORN https://uncn.jp

4. Japanese AdTech Industry

5. Japanese AdTech Industry
JP 2016 Internet Ads Market Size Research by CCI
http://www.cci.co.jp/news/release/2017_04_17/1.html

6. Japanese AdTech Industry
JP 2016 Internet Ads Market Size Research by CCI
http://www.cci.co.jp/news/release/2017_04_17/1.html
dentsu company

7. Japanese AdTech Industry
JP 2016 Internet Ads Market Size Research by CCI
http://www.cci.co.jp/news/release/2017_04_17/1.html
$10 Billion
Market
Smartphone
Shift
SmartphoneDesktop

8. Japanese AdTech Industry
https://www.slideshare.net/HiroshiKondo/jp-chaosmap-20152016

9. Japanese AdTech Industry
https://www.slideshare.net/HiroshiKondo/jp-chaosmap-20152016
Fragmented and Competitive
same as other regions

10. Japanese AdTech Community

11. Japanese AdTech Community
AdTech Meetup by AWS in 2016 #AWSAdTechJP

12. Japanese AdTech Community
AdTech Meetup by AWS in 2016 #AWSAdTechJP
“Digital Marketing”
Trend
DialogOne
“LINE” Business Connect
“AdNetwork”
Admin Tools
http://aws.typepad.com/sajp/2016/07/aws-adtech-jp.html
Wrap-up
Blog Post

13. Japanese AdTech Community
Akiba Lab – Over 800 people in Facebook group

14. Japanese AdTech Community
Akiba Lab – Over 800 people in Facebook group
アドテク⇒AdTech
Akiba Lab is a Japanese AdTech community
Big year-end party in Dec 2016
Lightning Talks

15. Japanese AdTech Community
#CTONight powered by AWS

16. Japanese AdTech Community
#CTONight powered by AWS
Japanese Top-Notch
CTOs gathering

17. Japanese AdTech Community
#CTONight powered by AWS
 AdTech CTOs at CTO Night 

18. Japanese AdTech Community
#CTONight powered by AWS
 AdTech CTOs at CTO Night 
IM-DMP FreakOut
VOYAGE
GROUP
Geniee

19. Japanese AdTech Community
AdTech CTOs Round Table
 Deeper AdTech Talks!

20. Japanese AdTech Community
AdTech CTOs Round Table
 Deeper AdTech Talks!
i-mobile
Speee
Intimate
Merger
Fringe81
FreakOut
SuperShip Geniee

21. Japanese AdTech on AWS
Use-Cases

22. AdTech on AWS Use-Cases in Japan
Dynalyst http://www.dynalyst.io
 Re-Targeting / Re-Engaging
 Japan and U.S.
fluct https://fluct.jp
 SSP: 30 billion impressions in a month
 Ajitofm: Tech Podcast @ inside company Bar
IM-DMP https://corp.intimatemerger.com
 Public DMP
 Small Team, Big Result
UNICORN https://uncn.jp
 Full Automated Marketing Platform
 International Engineers in Tokyo

23. CyberAgent - Internet Conglomerate
https://www.cyberagent.co.jp/en/

24. CyberAgent - Internet Conglomerate
http://pdf.cyberagent.co.jp/C4751/GEqq/Pzq8/RSrX.pdf

25. CyberAgent - Internet Conglomerate
http://pdf.cyberagent.co.jp/C4751/GEqq/Pzq8/RSrX.pdf
About 500 million USD
Ad Business

26. CyberAgent - AdTech Studio
https://adtech.cyberagent.io/
https://www.slideshare.net/InsightTechnology/dbts-
tokyo-2016-b15-bigdata-actian-matrix-mapr-cyberagent

27. CyberAgent - AdTech Studio
https://adtech.cyberagent.io/
https://www.slideshare.net/InsightTechnology/dbts-
tokyo-2016-b15-bigdata-actian-matrix-mapr-cyberagent

28. Dynalyst - Dynamic Retargeting for Game Apps
Massive Audience
Personalized Engagement
AWS Summit Tokyo 2014
AWS Summit Tokyo 2015

29. Dynalyst - Log Processing Architecture

30. Japan
US
ap-northeast-1
us-east-1
Up to
100 instances
Up to
80 shards
KCL on ECS
Docker Cluster
S3
Redshift
EMR
Up to
100 instances
Up to
80 shards
KCL on ECS
Docker Cluster
Dynalyst - Log Processing Architecture

31. Japan
US
ap-northeast-1
us-east-1
Up to
100 instances
Up to
80 shards
KCL on ECS
Docker Cluster
S3
Redshift
EMR
Up to
100 instances
Up to
80 shards
KCL on ECS
Docker Cluster
Dynalyst - Log Processing Architecture
Petabyte Scale

32. Dynalyst - Cross Region Replication
Real-Time Bidding
https://media.mopub.com/media/filer_public/30/1f/301ffdbc-1edb-4e8a-ab22-a3d5db57851e/mopub_dynalyst_case_study.pdf

33. Dynalyst - Cross Region Replication
Real-Time Bidding
https://media.mopub.com/media/filer_public/30/1f/301ffdbc-1edb-4e8a-ab22-a3d5db57851e/mopub_dynalyst_case_study.pdf
Network Latency is Critical
https://www.mopub.com/resources/mopub-demand/mopub-marketplace-overview/network-infrastructure/

34. Dynalyst - Cross Region Replication
DynamoDB Streams
Amazon RDS Aurora

35. Dynalyst - Real-Time Bidding
Train Model: Spark ML / Save Model: Redis
Quick Response to Bid Requests!
EMR ElastiCacheS3
Bid Request
Memcached
Redis Aurora
DynamoDB

36. Dynalyst - Go Global with AWS!
Shuhei Kimura
 Moving back and forth from Japan to U.S.
 Diving deeply into U.S. AdTech eco-system
 Planning to use another AWS region in US West

37. VOYAGE GROUP - fluct: SSP

38. fluct - Serverless Architecture in 2016
Serverless for Analyzing contents
Better Contents/Context matched Ad delivery
https://speakerdeck.com/suzuken/how-to-use-aws-lambda-in-document-processing-pipeline

39. fluct – SSP: 30billion impressions in a month
Kenta Suzuki
A. Advertising transparency
 Players are relying on each other
 Preventing unethical actions is an
entire industry problem!
 Let’s make the Internet better
place 
Q. What is the trend in AdTech
industry?

40. fluct magazine https://magazine.fluct.jp
fluct – SSP: 30billion impressions in a month

41. fluct – SSP: 30billion impressions in a month
“ads.txt” aims to increase transparency
in the AdTech ecosystem
How do we introduce ads.txt?
fluct magazine https://magazine.fluct.jp

42. fluct - Ad Verification Architecture

43. Ad
tag
ALB ECS
Lambda
Lambda
Amazon ES
3rd Party
service
Kinesis
fluct - Ad Verification Architecture

44. Ad
tag
ALB ECS
Lambda
Lambda
Amazon ES
3rd Party
service
Kinesis
Speed Layer
Batch Layer
fluct - Ad Verification Architecture

45. Ad
tag
ALB ECS
Lambda
Lambda
Amazon ES
3rd Party
service
Kinesis
Running “Golang” application
on AWS Lambda w/ Apex
fluct - Serverless Architecture

46. Apex – Serverless Infrastructure
“build, deploy, and manage AWS Lambda functions with ease”
http://apex.run/

47. https://speakerdeck.com/kakakakakku/case-study-of-lambda-and-apex
Apex – Serverless Infrastructure
Running Golang on AWS Lambda

48. https://speakerdeck.com/kakakakakku/case-study-of-lambda-and-apex
Running Golang on AWS Lambda
Apex – Serverless Infrastructure

49. Tech Podcast - VOYAGE GROUP
https://www.instagram.com/p/BXWzZ9ngYLZ/HUGO (https://gohugo.io/) + Hosting on S3
Ajitofm https://ajito.fm/

50. https://ajito.fm/2/
Tech Podcast - VOYAGE GROUP
Running Golang on AWS Lambda
 Node.js -> Golang
Running Golang as a Child Process
Utilize STDIN and STDOUT
Sounds like “CGI” in Cloud ERA...
http://www.kent-web.com/

51. https://ajito.fm/2/
 Node.js -> Golang
Running Golang as a Child Process
Utilize STDIN and STDOUT
Sounds like “CGI” in Cloud ERA...
Popular CGI Examples
In 90s…
Tech Podcast - VOYAGE GROUP
Running Golang on AWS Lambda
http://www.kent-web.com/

52. https://ajito.fm/2/
 Node.js -> Golang
Running Golang as a Child Process
Utilize STDIN and STDOUT
Sounds like “CGI” in Cloud ERA...
Tech Podcast - VOYAGE GROUP
Running Golang on AWS Lambda
http://www.kent-web.com/

53. AJITO – VOYAGE GROUP Inside Company Bar

54. AJITO – VOYAGE GROUP Inside Company Bar
Deeper tech-talks after business hours!
You wanna join, huh? 

55. Intimate Merger - IM-DMP

56. Intimate Merger - IM-DMP
Intimate Merger
 Founded in 2013 as a Joint Venture
FreakOut: The first DSP in Japan
Preferred Infrastructure: Cutting Edge Tech
 Shareholders in 2017
FreakOut Holdings: Global Marketing Tech group
Dentsu: World’s leading Advertising Agency
YJCapital: Yahoo! Japan Corporate Venture Capital

57. Intimate Merger - IM-DMP
w/ dentsu
 Contribute to Public DMP ”dPublic” by dentsu
w/ Yahoo! Japan
 Connect to Yahoo! Japan DMP
https://corp.intimatemerger.com/archives/1855/

58. Intimate Merger - IM-DMP
w/ dentsu
 Contribute to Public DMP ”dPublic” by dentsu
w/ Yahoo! Japan
 Connect to Yahoo! Japan DMP
https://corp.intimatemerger.com/archives/1855/
PsychographicDemographic
400 million
Audience Data

59. IM-DMP - Architecture

60. IM-DMP - Architecture
ECS
RDS
ElastiCache
Spot Fleet
ECS
CSV
TSV
JSON
SQS
IM-DMP
UI&API Data
Processing
S3
3rd Party
Partners

61. ECS
RDS
ElastiCache
CSV
TSV
JSON
SQS
IM-DMP
UI&API Data
Processing
S3
3rd Party
Data Providers
Everything is started from S3 upload
Simple File-based Trigger
Spot Fleet
ECS
IM-DMP - S3 File-based Architecture

62. ECS
RDS
ElastiCache
CSV
TSV
JSON
SQS
IM-DMP
UI&API Data
Processing
S3
3rd Party
Partners
ECS & Spot Fleet
Spot Fleet
ECS
IM-DMP - Amazon ECS & EC2 Spot Fleet

63. AWS Summit Tokyo 2017
https://www.youtube.com/watch?v=vJZJY_t8JDE https://www.slideshare.net/im_docs/amazon-ecs-spotfleet
IM-DMP - Amazon ECS & EC2 Spot Fleet

64. AWS Summit Tokyo 2017
https://www.youtube.com/watch?v=vJZJY_t8JDE https://www.slideshare.net/im_docs/amazon-ecs-spotfleet
IM-DMP - Amazon ECS & EC2 Spot Fleet

65. Amazon EC2 Spot Instances
July 26, 2017 / ap-northeast-1 / Linux
On
Demand
Reserved Instances for 1 year
Spot
Instances
Spot Block
All
Upfront
Partial
Upfront
No
Upfront 1h 6h
c4.large $0.126
$0.084
(33%)
$0.086
(32%)
$0.090
(29%)
$0.029
(77%)
$0.077
(39%)
$0.098
(22%)
m4.large $0.129
$0.081
(37%)
$0.083
(36%)
$0.087
(32%)
$0.027
(79%)
$0.101
(21%)
$0.128
(0.7%)
r3.large $0.20
$0.127
(36%)
$0.130
(35%)
$0.149
(26%)
$0.031
(84%)
$0.116
(42%)
$0.147
(26%)

66. On
Demand
Reserved Instances for 1 year
Spot
Instances
Spot Block
All
Upfront
Partial
Upfront
No
Upfront 1h 6h
c4.large $0.126
$0.084
(33%)
$0.086
(32%)
$0.090
(29%)
$0.029
(77%)
$0.077
(39%)
$0.098
(22%)
m4.large $0.129
$0.081
(37%)
$0.083
(36%)
$0.087
(32%)
$0.027
(79%)
$0.101
(21%)
$0.128
(0.7%)
r3.large $0.20
$0.127
(36%)
$0.130
(35%)
$0.149
(26%)
$0.031
(84%)
$0.116
(42%)
$0.147
(26%)
July 26, 2017 / ap-northeast-1 / Linux
Amazon EC2 Spot Instances

67. ECS
RDS
ElastiCache
CSV
TSV
JSON
SQS
IM-DMP
UI&API Data
Processing
S3
3rd Party
Partners
Spot Instances
Spot Fleet
ECS
IM-DMP - Elasticsearch on Spot Instances

68. Elasticsearch: Approx. 400 million IDs
 Extract IDs with
 Keyword (by browsing history)
 Segment
 User Agent
 IP address
 Geo
https://www.slideshare.net/im_docs/elasticsearch-48873206
IM-DMP - Elasticsearch on Spot Instances

69. Elasticsearch on Spot Instances
 approx. 500vCPUs for Analytics workload
Over 8vCPUs i3 Instances
IM-DMP - Elasticsearch on Spot Instances

70. Intimate Merger
Small engineering team delivers Big results!
Masafumi Osogoe & Kazuki Matsuda
Kazuki Masafumi

71. Bulbit - Adways company
http://adways.kr/https://www.adways.net

72. Bulbit - Party Track & UNICORN

73. 73
UNICORN - Full Automated Marketing Platform

74. Greatly Skilled Engineers from China 
Hailin Hu
Xiaoyi Qu
UNICORN - Full Automated Marketing Platform
 How do you feel about
working on AdTech in
Japan?
 What are you focusing on?
HailinXiaoyi

75. Greatly Skilled Engineers from China 
Hailin Hu
Xiaoyi Qu
UNICORN - Full Automated Marketing Platform
It’s like a “Gold Mine”
 Day-by-Day Evolution
 Achieving Goals with latest
Big Data Technologies
 Utilize “Amazon Athena”
in a massive way!
HailinXiaoyi

76. UNICORN - Architecture

77. Auto Scaling
Up to 200
instances
Athena
Redshift
Deep Learning
on EC2
S3
UNICORN - Architecture

78. Auto Scaling
Up to 200
instances
Athena
Redshift
Deep Learning
on EC2
S3
UNICORN - Real-Time Bidding
From Ruby to Golang
“Speed is King”
in Real-Time Bidding

79. Auto Scaling
Up to 200
instances
Athena
Redshift
Deep Learning
on EC2
S3
UNICORN - Data Analysis
 Extract data for Machine Learning every 30min
 Ad-Hoc Big Data Analysis

80. Auto Scaling
Up to 200
instances
Athena
Redshift
Deep Learning
on EC2
S3
UNICORN - Machine Learning
 w/ Minimum Libraries
 No Heavy Framework
 As Fast As Possible!!
 Making Steady Effort
 Plan-Do-Check-Act
 Parameter Tuning
 A/B Testing

81. Auto Scaling
Up to 200
instances
Athena
Redshift
Deep Learning
on EC2
S3
UNICORN - Machine Learning
For Real-Time Bidding,
Bidding servers load “Trained Models” into Memory

82. Auto Scaling
Up to 200
instances
Athena
Redshift
Deep Learning
on EC2
S3
UNICORN - Big Data Technology
 Right Technology in the Right Place
 Recently in favor with “Apache Flink”

83. AdTech use-cases in Japan

84. 감사합니다!!

85. Let's meet up at re:Invent 2017!!
https://reinvent.awsevents.com/

86. Let's meet up at re:Invent 2017!!
https://reinvent.awsevents.com/

87. Let's meet up at re:Invent 2017!!
https://reinvent.awsevents.com/
@Kimchi Korean BBQ
http://www.kimchilasvegas.com/