AI has already been integrated into many use cases, but we’ve just scratched the surface of what’s possible. In this session, we cover how to use the AWS AI services to tackle three use cases that can deliver immediate value: 1) “voice of the customer” analytics to better understand what your customers are thinking and saying, 2) document analysis and processing to move beyond the limitations of traditional OCR, and 3) chatbots to improve in-app customer service and customer contact center experiences. We also discuss how to use AI in use cases within the Media, Healthcare, and Financial Services industries.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Adding intelligence to applications
Randall Hunt
Sr. technical evangelist and software engineer
@jrhunt
randhunt@amazon.com
A I M 2 0 3

2. Our mission at AWS
Put machine learning (ML) in the hands
of every developer

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Why Amazon Web Services (AWS) for artificial intelligence (AI)?
Broadest and deepest set of
AI and ML services for your
business
Solving the toughest
problems holding back ML:
cost, ease of use, and data
75% lower inference cost
70% cost reduction in data
labeling
“No ML experience required”
Unmatched support for
the most popular
frameworks
Built on the most comprehensive
cloud platform optimized for ML
85% of TensorFlow projects
in the cloud happen on
AWS
200+ new features and
services since January
2018
AWS holds the top spots on
Stanford’s benchmark for fastest
training time, lowest cost, lowest
inference latency
Deepest set of security and
encryption features, with robust
analytics capabilities

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
More ML happens on AWS than anywhere else

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Frameworks and interfaces
ML for data scientists
KERAS
Frameworks Interfaces
Application services
ML for everyone
Infrastructure
Powering ML
A m a z o n
L e x
Amazon Rekognition
Video and Image
A m a z o n
T r a n s c r i b e
A m a z o n
T r a n s l a t e
A m a z o n
C o m p r e h e n d
A m a z o n
F o r e c a s t
A m a z o n
P e r s o n a l i z e
Amazon ML
AMIs
AWS IoT
Greengrass ML
Inference
Compute
NVIDIA Tesla V100 GPUs EC2 C5 FPGAs Amazon Elastic
Inference
A m a z o n
P o l l y
A m a z o n
T e x t r a c t
Vision Speech Language Chatbots Forecasting Recommendations
Amazon MachineLearning(Amazon ML) stack:Broadestand deepestset of capabilities
Platform services
ML for engineers
Amazon SageMaker
B u i l d T r a i n D e p l o y
Prebuilt algorithms and notebooks One-click model training and tuning One-click deployment and hosting
Data labeling (Amazon SageMaker Ground Truth) Optimization (Amazon SageMaker Neo) AWS Auto Scaling
AWS Marketplace Reinforcement learning Amazon Elastic Inference integration
Hyperparameter optimization Amazon VPC and AWS PrivateLink

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Pretrained AI services that require no
ML skills or training
Easily add intelligence to your existing
apps and workflows
Quality and accuracy from
continuously learning APIs
Application services
ML for everyone
Amazon
L e x
Amazon Rekognition
video and image
Amazon
T r a n s c r i b e
Amazon
T r a n s l a t e
Amazon
C o m p r e h e n d
Amazon
F o r e c a s t
Amazon
P e r s o n a l i z e
Amazon
P o l l y
Amazon
T e x t r a c t
Vision Speech Language Chatbots Forecasting Recommendations

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Documents are important
Primary tool of record keeping, communicating, collaborating, and transacting
Finance
Insurance
Real estate
Accounting
Tax management
Medical
Legal
Business management
Education
And many more…

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
16.3 million US mortgage applications ($2.1 trillion) in 2016
*

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
About 240 million W-2 tax forms will be processed for FY2018
in the United States
* https://www.irs.gov/individuals/w-2-verification-code

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Need for processing documents

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
How documents are processed today
Optical character recognition
(OCR)
Manual
processing
Rules and
template-based extraction

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Challenges for processing documents
Manual processing
Expensive Error-prone Time-consuming

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Challenges for processing documents
Manual processing
Output
1. Exempt is true
2. 28 is true
3. CPP/QPP is true
4. RPC/RRQ is true

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Challenges for processing documents
OCR
Error-prone Flat bag of wordsSimple documents only

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Challenges for processing documents
OCR
Output
Extract data quickly
No code templates to accurately maintain

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Challenges for processing documents
OCR
Output
Start Date End Date Employer Name Position Held Reason for leaving
1/15/2009 6/30/2013 Any Company Head Baker Family Relocated

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Challenges for processing documents
Output
Full Name Date of Birth Gender
John X Doe 01 01 1971
Male
First Middle Last MM DD YYYY
Female

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Challenges for processing documents
Rules and template-based extraction
Limited by
accuracy of OCR
Significant development and
management overhead
Templates
are brittle

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Challenges for processing documents
Rules and template-based extraction
The well-known W-2 US tax form has hundreds of variants each year

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
It looks easy, but…
…not asingle correspondingpixel value in common

21. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OCR++ service to easily extract text and data from virtually
any document. No ML experience required.

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Put AI to work, no ML experience required
Increase efficiency with automated document analysis
OCR | Automatic data extraction | Natural language processing |
Intelligent search | Text analytics
A m a z o n C o m p r e h e n d a n d
A m a z o n C o m p r e h e n d M e d i c a l
A m a z o n
T e x t r a c t

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Textract features
Text extraction Table extraction Form extraction

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Textract—Text Extraction
Blocks: page, paragraph, line, word
is washed by waves, and cooled

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Name Description
Blocks List of blocks identified
from the document
ID Unique ID of the unit
Relationships Child
Block type Page, paragraph, line, word
Pages Contains number of
pages in the document
Amazon Textract—Text extraction API
DetectDocumentText
Name Description
Document Blob or Amazon Simple
Storage Service (Amazon S3)
object

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Textract—Table Extraction
Blocks: page, table, cell
For each “block” you get:
• Text
• Confidence score
• Block relationships (e.g., cells within a table)

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Name Description
Document Blob or Amazon S3 object
FeatureTypes Tables
Name Description
Blocks List of blocks identified
from the document
ID Unique ID of the unit
Relationships Child
Block type Page, table, cell
Pages Contains number of
pages in the document
Amazon Textract—Table Extraction API
AnalyzeDocument with “table” asFeatureTypesparameter

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Textract—Form Extraction
Blocks: page, key_value_set
For each “block” of your document:
• Form field name (key) and field value (value) association
• Confidence score
• Page number
• Block relationships

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Name Description
Document Blob or Amazon S3 object
FeatureTypes Forms
Name Description
Blocks List of blocks identified
from the document
ID Unique ID of the unit
Relationships Key, value, child
Block type Page, key_value_set
Pages Contains number of
pages in the document
Amazon Textract—Forms Extraction API
AnalyzeDocument with “forms” asFeatureTypesparameter

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Supports single-page documents
such as images (e.g., mobile
capture)
For multipage documents
up to 3,000 pages
Amazon Textract—Sync and async

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Textract—Text extraction simplified
Output
Extract data quickly and
accurately
No code or templates to
maintain

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Output {
Start Date: 1/15/2009
End Date: 6/30/2013
Employer Name: Any Company
Position Held: Head Baker
Reason for leaving: Family relocated
}
Amazon Textract—Table extraction simplified

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Output Full Name:
First: John
Middle: X
Last: Doe
Date of Birth:
MM: 01
DD: 01
YYYY: 1971
Gender:
Male: True
Female: False
Amazon Textract—Form extraction simplified

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Reference architecture—Index and search documents
Input
Uploaded document
images such as tax
forms, credit
applications, or
medical notes
Amazon S3
Uploaded
documents are
stored in data lake
AWS Lambda
A Lambda function is
triggered to initiate
document analysis
using the
Hieroglyph API
Amazon Textract
Automatically
extract text,
including key-value
pairs and tables
Amazon
Elasticsearch
Service
(Amazon ES)
Extracted data and
confidence scores
are indexed to
enable document
search
Output
Perform contextual
search on millions of
documents or
integrate data into
your document
management system

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Reference architecture—Form capture
Input
Customer uses mobile
app to capture a photo of
a W-2 form
Amazon Textract
Amazon Textract API is
integrated into end-user
application to
automatically extract text
from W-2 form and
auto-populate form fields
Customer Application
Customers experience
real-time capture of their
tax information by taking
a photo instead of
manual data entry
Database
User submitted data is
loaded into a
database for use in tax
preparation

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Reference architecture—Extract for natural language
processing (NLP)
Quicklyturnextracted text/datainto actionableinsights
Input
Uploaded document
images of medical
notes, explanation of
benefits, and
patient forms
Amazon S3
Uploaded
documents are
stored in Amazon S3
NLP
Use NLP to extract
insights from
medical documents
Amazon ES
Easily search
through extracted
data and text
insights
Output
Discover medical
insights to improve
patient care
Amazon Textract
Automatically
extract words, lines
of text, and tables

39. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Discover insights and relationships in text

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Polly
Use cases
Content creation
Mobile & desktop applications
Internet of Things (IoT)
Education and e-learning
Telephony
Game development
Key features
52 voices across 25 languages
Lip-syncing & TEXT HIGHLIGHTING
Fine-grained VOICE CONTROL
Custom VOCABULARIES
Available in 14 AWS Regions

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Transcribe
Automatic speech recognition
Time
stamps
Support for
both regular and
telephony audio
Punctuation
and
formatting
Amazon S3
integration
Recognize
multiple
speakers
Custom
dictionaries
? !

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Transcribe
Use cases
Call centers Subtitles for video
on demand
Transcribe meetingsBroadcast closed
captions

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
21 languages
417 Combinations
Key features
Real-time
< 500 ms/sentence on average
< 150 ms/conversational/short form
Tag Handling
XML tags placement maintains styling
and formatting through translation
< / >
Data security
Data ownership
Encryption
Access management
Ease of use
Simple API calls and partner solutions
$15/1 million characters
or $0.000075 per word;
pay as you go, 2-million characters
monthly free tier
HIPAA Eligible
Amazon Translate
Neural machine translation

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Comprehend
Natural-language processing
Sentiment Entities LanguagesKey phrases Topic modeling

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Comprehend
Discover insights and relationships in text
Entities
Key phrases
Language
Sentiment
Syntax
Grouping
A m a z o n C o m p r e h e n d

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Comprehend
Use cases
Voice of customer analytics Semantic search Knowledge management/discovery
Analyzing what customers are saying about your
brand, products, and services
Making search smarter by searching on key
phrase, sentiment, and topic
Organizing documents, categorizing by topic
and personalizing experiences

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Put AI to work, no ML experience required
Understand the voice of your customer
Sentiment analysis | App localization | Translation services | Transcription services | Cataloging media | Accessibility
A m a z o n
T r a n s c r i b e
A m a z o n
T r a n s l a t e
A m a z o n
C o m p r e h e n d
A m a z o n
R e k o g n i t i o n
I m a g e
A m a z o n
R e k o g n i t i o n
V I D E O

48. Targeted
customer acquisition
VidMob uses Amazon Rekognition and
Amazon Transcribe for metadata extraction
and sentiment analysis, to help marketers understand
which videos resonate with audiences. This allows
marketers to promote targeted content to acquire new
customers.

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Detecting Customer Sentiment
Bringing it together: Multilanguage product analytics
Twitter Stream
API
Amazon S3 Amazon AthenaAmazon Kinesis Amazon
Translate
Lambda
Amazon
Comprehend
Amazon
Transcribe

50. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Conversational interfaces for your applications powered by
the same deep-learning technologies as Alexa

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Lex
Scale
Business logic
SecurityAnalytics
Text to speech
Speech to intent
End to
end
Native support &
maintains context
One-click deployment
Completely managed
service
Native integration with
Lambda
Encrypted data in
transit and at rest
Monitor and
improve
Amazon Polly integrated
into API
ASR + NLU integrated
into one API
Dialog management Deployment

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Designed for builders
Efficient and intuitive tools to build conversations

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Lex
Utterances
Spoken or typed phrases that invoke your
intent
BookHotel
Intents
An intent performs an action in response
to natural-language user input
Slots
Slots are input data required to fulfill the
intent
Fulfillment
Fulfillment mechanism for your intent

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Lex—Technology
Amazon Lex
Automatic speech
recognition (ASR)
Natural-language
understanding (NLU)
Same technology that powers Alexa
Amazon Cognito AWS CloudTrail Amazon CloudWatch
AWS Services
Action
Lambda
Authentication and
visibility
Speech
API
Language
API
Fulfillment
End users
Developers
Console
SDK
Intents,
slots,
prompts,
utterances
Input:
Speech
or text
Multiplatform clients:
mobile, IoT, web,
chat
API
Output:
Speech (via Amazon Polly TTS)
or text

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Amazon Lex
Use cases
Informational
bots
Chatbots for everyday
consumer requests
Application
bots
Build powerful
interfaces to mobile
applications
News updates
Weather information
Game scores….
Book tickets
Order food
Manage bank accounts….
Enterprise productivity bots
Streamline enterprise
work activities and
improve efficiencies
Check sales numbers
Marketing performance
Inventory status….
IoT bots
Enable conversational interfaces
for device interactions
Wearables
Appliances
Auto….
Contact center
bots
Chatbots for customer
service IVR
Account inquiries
Bill payment
Service update….

56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Put AI to work, no ML experience required
Modernize your contact center to improve customer service
Conversational chat bots | Call transcription | Intelligent routing | Sentiment analysis |
VoC analytics text-to speech | Multilingual omni-channel communication
A m a z o n
P o l l y
A m a z o n
T r a n s c r i b e
A m a z o n
T r a n s l a t e
A m a z o n
C o m p r e h e n d
A m a z o n
L e x

57. Meaningful
customer interactions
Liberty Mutual uses Amazon Lex and AI services
to develop natural-language-driven conversational apps
to allow customer service agents to respond to
customer requests with real-time and contextual
intelligence, improving response time and quality of
service.

58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Improving contact centers with AI
Amazon
Athena

60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Frameworks and interfaces
ML for data scientists
KERAS
Frameworks Interfaces
Application services
ML for everyone
Platform services
ML for engineers
Infrastructure
Powering ML
A m a z o n
L e x
Amazon Rekognition
Video and Image
A m a z o n
T r a n s c r i b e
A m a z o n
T r a n s l a t e
A m a z o n
C o m p r e h e n d
A m a z o n
F o r e c a s t
A m a z o n
P e r s o n a l i z e
Amazon ML
AMIs
AWS IoT
Greengrass ML
Inference
Compute
NVIDIA Tesla V100 GPUs EC2 C5 FPGAs Amazon Elastic
inference
A m a z o n
P o l l y
A m a z o n
T e x t r a c t
Vision Speech Language Chatbots Forecasting Recommendations
Amazon SageMaker
B u i l d T r a i n D e p l o y
Prebuilt algorithms and notebooks One-click model training and tuning One-click deployment and hosting
Data labeling (Ground Truth) Optimization (Amazon SageMaker Neo) AWS Auto Scaling
AWS Marketplace Reinforcement learning Amazon Elastic Inference integration
Hyperparameter optimization Amazon VPC and AWS PrivateLink
Amazon ML stack: Broadest and deepest set of capabilities

61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS ML training and certification

62. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS DeepLens
• Purpose-built for ML-skills development
• Fully programmable and customizable
• Build custom Amazon SageMaker models
• 10-minutes to your first deep-learning project
The world’s first deep-learning-enabled video camera for developers

63. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS DeepRacer
• Build machine-learning models in Amazon SageMaker
• Train, test, and iterate on the track using the AWS
DeepRacer 3-D racing simulator
• Compete in the world’s first global autonomous racing
league to race for prizes and a chance to advance to win
the coveted AWS DeepRacer Cup
A fully autonomous 1/18th-scale race car designed to help you learn about reinforcement learning
through autonomous driving

64. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Randall Hunt
@jrhunt
randhunt@amazon.com