1. S U M M I T
SYDNEY

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Building an immersive, interactive customer
experience using artificial intelligence and
augmented reality
Injae Kwak
Solutions Architect
Amazon Web Services

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
• Primer on XR (Augmented Reality/Virtual Reality)
• Building interactive smart applications
• Combining AI/ML and AR
• Getting started with AWS services
• Variety showcase of demos

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
To understand Augmented Reality
We must go back..

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Great Scott!

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T https://www.facebook.com/groups/sydneypokemongo

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The real world Fully immersive
Digital information
layered over the
real world
Virtual objects
interacting with real
world
Blend of digital and
real world info to
enhance reality
(we are here!)

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Web ImmersiveMobile Cloud
Evolution of User Interfaces

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How customers are using XR
Instructional design
Brand engagement
Data visualisation
Simulation
Virtual showcases
Virtual tours
Digital signage Customer Service
agents & chatbots

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
XR customers across industries

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What are some of the challenges?

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Sumerian

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Leverage rich library of assets

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Entities Panel
Canvas
Assets Panel
State Graph
WebVR Mode
Components Panel

15. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Building interactive experience through AI/ML

17. M L F R A M E W O R K S &
I N F R A S T R U C T U R E
The Amazon AI/ML Stack
A I S E R V I C E S
A M A Z O N
R E K O G N I T I O N
I M A G E
A M A Z O N
P O L L Y
A M A Z O N
T R A N S C R I B E
A M A Z O N
T R A N S L A T E
A M A Z O N
C O M P R E H E N D
C O M P R E H E N D
M E D I C A L
A M A Z O N
L E X
A M A Z O N
R E K O G N I T I O N
V I D E O
Vision Speech Chatbots
A M A Z O N S A G E M A K E R
B U I L D T R A I N
A M A Z O N
F O R E C A S T
A M A Z O N
T E X T R A C T
A M A Z O N
P E R S O N A L I Z E
D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimisation ( N E O )
One-click deployment & hosting
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
E C 2 P 3
& P 3 d n
E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C
I N F E R E N C E
Models without training data (REINFORCEMENT LEARNING)
Algorithms & models ( A W S M A R K E T P L A C E )
Language Forecasting Recommendations
NEW
NEWNEW
NEW
NEW
NEW NEWNEWNEW

18. M L F R A M E W O R K S &
I N F R A S T R U C T U R E
The Amazon AI/ML Stack
A I S E R V I C E S
A M A Z O N
R E K O G N I T I O N
I M A G E
A M A Z O N
P O L L Y
A M A Z O N
T R A N S C R I B E
A M A Z O N
T R A N S L A T E
A M A Z O N
L E X
A M A Z O N
R E K O G N I T I O N
V I D E O
Vision Speech Chatbots
A M A Z O N S A G E M A K E R
B U I L D T R A I N
A M A Z O N
F O R E C A S T
A M A Z O N
T E X T R A C T
A M A Z O N
P E R S O N A L I Z E
D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization ( N E O )
One-click deployment & hosting
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
E C 2 P 3
& P 3 d n
E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C
I N F E R E N C E
Models without training data (REINFORCEMENT LEARNING)
Algorithms & models ( A W S M A R K E T P L A C E )
Language Forecasting Recommendations
NEW
NEWNEW
NEW
NEW
Pre-trained AI services that require
no ML skills or training
Easily add intelligence to your
existing apps and workflows
Quality and accuracy from
continuously-learning APIs
A M A Z O N
C O M P R E H E N D
C O M P R E H E N D
M E D I C A L
NEW NEWNEWNEW

19. M L F R A M E W O R K S &
I N F R A S T R U C T U R E
The Amazon AI/ML Stack
A I S E R V I C E S
A M A Z O N
R E K O G N I T I O N
I M A G E
A M A Z O N
P O L L Y
A M A Z O N
T R A N S C R I B E
A M A Z O N
T R A N S L A T E
A M A Z O N
L E X
A M A Z O N
R E K O G N I T I O N
V I D E O
Vision Speech Chatbots
A M A Z O N S A G E M A K E R
B U I L D T R A I N
A M A Z O N
F O R E C A S T
A M A Z O N
T E X T R A C T
A M A Z O N
P E R S O N A L I Z E
D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization ( N E O )
One-click deployment & hosting
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
E C 2 P 3
& P 3 d n
E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C
I N F E R E N C E
Models without training data (REINFORCEMENT LEARNING)
Algorithms & models ( A W S M A R K E T P L A C E )
Language Forecasting Recommendations
NEW
NEWNEW
NEW
NEW
A M A Z O N
R E K O G N I T I O N
I M A G E
Object, scene, and activity detection
Facial recognition
Facial analysis
Pathing people movement
Unsafe content detection
Celebrity detection
Text extraction
A M A Z O N
C O M P R E H E N D
C O M P R E H E N D
M E D I C A L
NEW NEWNEWNEW

20. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Application architecture
Mobile client
Amazon Cognito
Amazon Rekognition
Authentication
Model Inference
User App
Object Detection App

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Customers using Amazon Rekognition

23. M L F R A M E W O R K S &
I N F R A S T R U C T U R E
The Amazon AI/ML Stack
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D
C O M P R E H E N D
M E D I C A L
L E XR E K O G N I T I O N
V I D E O
Vision Speech Chatbots
A M A Z O N S A G E M A K E R
B U I L D T R A I N
F O R E C A S TT E X T R A C T P E R S O N A L I S E
D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization ( N E O )
One-click deployment & hosting
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
E C 2 P 3
& P 3 d n
E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C
I N F E R E N C E
Models without training data (REINFORCEMENT LEARNING)
Algorithms & models ( A W S M A R K E T P L A C E )
Language Forecasting Recommendations
NEW NEWNEW
NEW
NEW
NEWNEW
NEW
NEW

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Using a custom machine learning model
Use Case: Sign Language to English Translation
Handshape
Position
Movement
Non-Manual Markers
Context

25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Application architecture
Mobile client
Amazon Cognito
Amazon Rekognition
Authentication
Model Inference
User App
Object Detection App
Mobile client
Amazon Cognito
Amazon SageMaker
Authentication
Model Inference
User App
Sign Language Translation App

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What about combining both together?
+
Augmented
Reality
Artificial
Intelligence
= ? ? ?

28. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
So how do I get started?

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Introducing AWS Amplify
XR
Authentication
Analytics
Pub/Sub
APIs

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dream Big
• Drive innovation through imagination
• Disrupt customer experience using emerging technologies
Start Small
• Add interactive experiences through AWS AI/ML and Amazon Sumerian
• Integrate rich functionalities through AWS Amplify
Build Fast
• Links to Tutorials and labs by tapping badge @ exit
• Download Demos by tapping badge @ exit

32. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Injae Kwak
Solutions Architect