1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
C I T Y N A M E
03.25.19
Machine Learning and IoT on AWS
using Amazon Go as a customer case study
Young Yang,
Machine Learning Specialist Solutions Architect
beyoung@amazon.com
S e s s i o n I D

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AGENDA
• Amazon Go – Why we start this and what are
challenges
• Retails Trends – Why AI + IoT are important?
• AWS AI and IoT – Tips for AI + IoT
© 2019 Amazon Web Services, Inc or its Affiliates. All rights reserved.

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why Did
We Build
Amazon Go?

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A simple equation…
+ =?

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A simple equation…
+ =?

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Intra Class Variations
Logos from the same brand have variations in shapes, sizes, colors and
textures on which they appear

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distortions and Occlusions
Logos are not always presented in their canonical form. There can be perspective
distortions, warping and occlusions

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A simple equation…
+ =?

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sensor
Management
Streaming
Services
Sensor
Fusion Vision
Algorithms Session
Management ....
Who took what?

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Go
Focus on unique value
Lower cost
Faster time-to-market
Lower operational burden
Partnership
Kinesis Video Streams
Amazon Go as early adopter
Real-world application
Production scale

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Video is critical to many applications
Smart Home
Security
MonitoringSmart City
Industrial Automation Computer Vision

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building a video system is complex
• Cumbersome to rewrite code for various devices and
development environments
• Difficult to “elastically” scale to millions of devices
• Requires reliable support for cadence, latency, and jitter on
streams
• Requires secure streaming and storage by default
• Need easy-to-use APIs to retrieve, replay, and process video at
any scale

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Stream video from millions of devices
Easily build vision-enabled apps
Playback in live or on-demand mode
Durable, searchable storage
Secure and Fully managed
Amazon Kinesis Video Streams
Stream video in real-time for playback, storage, and analytics
Kinesis Video
Streams

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kinesis Video Streams Inference Template for Amazon SageMaker (KIT)
Sample, parse, decode, invoke SageMaker in real-time

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML and on-lines aren’t disrupting retails…
customer expectations are!

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New Solution Opportunities
Retail Machine Learning
ML-based models for standard retail
requirements like dynamic pricing, product
recommendations
Video Analytics
Rekognition based customer identification
for traffic counting, sentiment analysis,
loss prevention, loyalty programs

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
/ ML based security
/ ML loss prevention & safety
/ ML planogram compliance
/ ML based employee task management
/ ML facilities monitoring & management
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Computer Vision

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Retail Trends
Demographic
Change
Different shopping
expectations
Stores Closing
.. and opening,
and transforming
Mass
Personalization
Curated product
choices based on
data insights
Direct to Consumer
(D2C)
Manufacturers seek
intimate customer
relationships

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Retail Trends
Cloud as the new
normal
Public cloud
adoption growing
Big Data and
Analytics
Customer and
operational insights
Global Data
Protection
Regulation
Data protection
regulation in 2018
Mixed Reality
Ubiquitous both in
digital and physical

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Just Walk Out Shopping
(Amazon Go store)
1-Click Checkout
(Amazon.com)
Frictionless Experiences

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ID by login cookie
(Amazon.com)
ID by license plate
(Pickup at Amazon Fresh store)
Frictionless Experiences

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customers Who Bought…
(Amazon.com)
If You Like / You’ll Love (Amazon
Books store)
Machine Learning

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Social proof
• Merchandising based on data
science, not intuition
• Real-time reviews
• Discovery experience
Inspired by Customers

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Highly Rated 4.8 Stars and Above
• Page Turners: Books Kindle Readers Finish in 3 Days or Less
• Book Club Picks from Goodreads
• Most Wished for Books
• If You Like…
• Read Local Section
• Kids Section
• Amazon Mobile Checkout
• Amazon Device Test Drive

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customers Don’t Buy Only “IoT”
Customers Buy “Services”

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Digital Transformation Drives IoT

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Retail Digital Transformation
Retail is in the midst of one of the most profound
transformations in its history driven by technology and
the convergence of People, Processes, Things, and Data.
People Processes Things Data

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tips For AI + IoT
Tip 1: Successful AI + IoT
projects require Exec support.
Need to be having business
outcomes, not just technical,
and discussions.

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New services &
business models
Products that get
better with time
Better relationship
with customers
Increased
efficiency
Intelligent decision
making
Data driven
discipline
Business outcomes with AI + IoT
Revenue growth
IoT data drives business growth
Operational efficiency
IoT data decreases OpEx

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS IoT architecture

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS IoT architecture

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS IoT Events
IoT virtuous cycle
AWS IoT Analytics
AWS IoT SiteWise
AWS IoT Device
Management
AWS IoT
Things Graph
AWS IoT Core
AWS IoT
Device Defender
AWS IoT
Device SDK
Amazon FreeRTOS
AWS IoT
Device Tester
AWS IoT
Greengrass

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tips For AI + IoT
Tip 2: This Is Not A Single
Service. Not Just AI + IoT.
Solve Problems and Create
Customer Values.

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Do Machines Actually Get Smarter?
AI / ML and IoT

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
IoT & Edge
ML / AIAnalytics

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
M L F R A M E W OR K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
REKOGNITION
IMAGE
POLLY TRANSCRIBE TRANSLATE COMPREHEND LEXREKOGNITION
VIDEO
Vision Speech Language Chatbots
A M A Z O N
S A G E M A K E R
B U I L D T R A I N
FORECAST
Forecasting
TEXTRACT PERSONALIZE
Recommendations
D E P L O Y
Pre-built algorithms & notebooks
Data labeling (GROUND TRUTH)
One-click model training & tuning
Optimization (N E O )
One-click deployment &
hosting
M L S E R V I C E S
Frameworks Interfaces Infrastructure
EC2 P3
& P3dn
EC2 C5 FPGAs GREENGRASS ELASTIC
INFERENCE
Reinforcement learning
Algorithms & models ( AWS MARKETPLACE
FOR MACHINE LEARNING)
A W S
D E E P R A C E R
A W S
D e e p L e n s
Amazon
Inferentia
Optimized

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML At The Edge

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“AIoT” (AI and IoT) — ML at the Edge
Train in the cloud, infer at the edge
Amazon SageMaker
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT Core
AWS IoT Analytics

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tips For AI + IoT
Tip 3: There is no compression
algorithm for experience. Learn
by doing, avoid analysis
paralysis.

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
HOW WE CAN HELP
• Brainstorming
• Custom modeling
• Training
• Work side-by-side with Amazon experts
ML Solutions Lab
• Practical education on ML for new
and experienced practitioners
• Based on the same material used to
train Amazon developers
Machine Learning Training
and Certification

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AGENDA
• Amazon Go – Why we start this and what are
challenges
• Retails Trends – Why AI + IoT are important?
• AWS AI and IoT – Tips for AI + IoT
© 2019 Amazon Web Services, Inc or its Affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A simple equation…
+ =?
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sensor
Management
Streaming
Services
Sensor
Fusion Vision
Algorithms
Session
Management ....
Who took what?
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kinesis Video Streams Inference Template for Amazon SageMaker (KIT)
Sample, parse, decode, invoke SageMaker in real-time

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML and on-lines aren’t disrupting retails…
customer expectations are!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
/ ML based security
/ ML loss prevention & safety
/ ML planogram compliance
/ ML based employee task management
/ ML facilities monitoring & management
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Computer Vision
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Retail Trends
Cloud as the new
normal
Public cloud
adoption growing
Big Data and
Analytics
Customer and
operational insights
Global Data
Protection
Regulation
Data protection
regulation in 2018
Mixed Reality
Ubiquitous both in
digital and physical
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Social proof
• Merchandising based on data
science, not intuition
• Real-time reviews
• Discovery experience
Inspired by Customers

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tips For AI + IoT
Tip 1: Successful AI + IoT
projects require Exec support.
Need to be having business
outcomes, not just technical,
and discussions.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tips For AI + IoT
Tip 2: This Is Not A Single
Service. Not Just AI + IoT.
Solve Problems and Create
Customer Values.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tips For AI + IoT
Tip 3: There is no compression
algorithm for experience. Learn
by doing, avoid analysis
paralysis.

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
NextGen Retail Store: Sensor Fusion
Predictive traffic counter
Video analytics
Aware digital signage
Just Walk Out
No Back Office
AI Merchandising
Wise Concierge
Beacons
Smart Inventory Mgmt
Smart Planogram
Aware Instore Signage
Intelligent Loss Prevention
Aware Music
Connected Store Controls
Retail IOT Managed Service
- Minimal installation
- Automatic scaling
- OTA updates
- Pay as you go
Greengrass

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Young Yang,
Machine Learning Specialist Solutions Architect
beyoung@amazon.com

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.