AWS Machine Learning Week at the San Francisco Loft: Automate for Efficiency with Amazon Transcribe & Amazon Translate Teaching a computer how to understand human language is one of the most challenging problems in computer science. However, significant progress has been made in automatic speech recognition (ASR) and machine translation (MT) to create highly accurate and fluent transcriptions and translations. Amazon Transcribe is an ASR service that makes it easy for developers to add speech to text capability to their applications, and Amazon Translate is a MT service that delivers fast, high-quality, and affordable language translation. In this session, you’ll learn how to weave machine translation and transcription into your workflows, to increase the efficiency and reach of your operations. Speaker: Martin Schade - R&D Engineer, AWS

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
Automate for Efficiency with Amazon Transcribe & Amazon
Translate
Speaker
Title

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
The AWS Machine Learning Stack
AI SERVICES
ML PLATFORMS
ML FRAMEWORKS
VISION
Rekognition
Video
Rekognition
SPEECH
TranscribePolly
LANGUAGE
ComprehendTranslate
CHATBOTS
Lex
AWS DeepLensAmazon SageMaker
TensorFlow MXNet PyTorch Caffe2 Chainer Horvod Gluon Keras
Mechanical Turk

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Transcribe

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Unprecedented rate of audio & video data generation
Audio
• Contact centers
• Meetings
Video
• Broadcast TV
• On-demand video
• Social media

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Solution
Transcribe
Customer
insights
Greater access
Global reach
Targeted
advertising
Search archives

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Transcribe - Automatic Speech Recognition
Support for
telephony
audio
Timestamp
generation
Intelligent
punctuation and
formatting
Recognize
multiple
speakers
Custom
vocabulary
Multiple
languages

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Support for a Wide Range of Audio/Video
Fidelity Format
Accurate transcription for 8 kHz
(telephony) audio or better
Multiple formats like FLAC,
WAV, MP3, and MP4

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Timestamping and Confidence Score
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 11.0 12.0 13.0 14.0 15.0 16.0 17.0
“Hi I would like to reschedule my flight to Seattle to later tonight”
3.480 sec
Confidence : 1
6.402 sec
Confidence : 0.95
10.541 sec
Confidence : 1

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Punctuation Support
please extrapolate the projections based on market
growth and segment share can you email it to me once
you’re done
Please extrapolate the projections based on market
growth and segment share. Can you email it to me once
you’re done?

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Speaker Attribution
Speaker 1
We are speaking with the
customer tomorrow.
Speaker 2
What’s our plan for the
meeting?
Speaker 3
I have it covered.
We are speaking with the
customer tomorrow.
What’s our plan for the
meeting?
I have it covered.
Speaker 1
Speaker 2
Speaker 3

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Custom Vocabulary
Upload custom
vocabulary
Higher
accuracy

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Transcribe: Use Cases
Call centers
Transcribe 8 kHz call
recordings with high
accuracy
Analyze the text with
Amazon Comprehend
Visualize results with
Amazon QuickSight

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Transcribe: Use Cases
Subtitles for VOD
Transcribe the
audio/video to text
Align the text with
the video based on
time stamp
Post-edit/ QA Translate to
language of choice
Broadcast closed captions

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Transcribe: Use Cases
Transcribe meetings
Transcribe the
recordings
Index text using
Amazon ES
Search and
discover content
on demand
Save recordings
to Amazon S3

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Transcribe: How It Works
Deep Learning based
Speech Recognition
Pre-Processing
Post-Processing
Audio
Transcription

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Transcribe Demo

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
A Sample Transcript – Accurate & Punctuated
Hello Harry I see you have a flight to Seattle tonight is
that what you’re calling about?
Yes I was hoping I could get a seat upgrade.
Sure thing I see you are a platinum member and there
is a seat in first class number 1C available. Does that
work?
That would be great.
Ok. You are all set and I have upgraded your seat to
1C. Is there anything else I can help with?
No thank you so much. Bye.

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Entities
Hello Harry I see you have a flight to Seattle tonight is
that what you’re calling about?
Yes I was hoping I could get a seat upgrade
Sure thing! I see you are a platinum member and there
is a seat in first class number 1C. available Does that
work?
That would be great!
Ok. You are all set and I have upgraded your seat to
1C. Is there anything else I can help with?
No thank you so much. Bye!

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Entities + Key Phrases
Hello Harry I see you have a flight to Seattle tonight is
that what you’re calling about?
Yes I was hoping I could get a seat upgrade
Sure thing! I see you are a platinum member and there
is a seat in first class number 1C. available Does that
work?
That would be great!
Ok. You are all set and I have upgraded your seat to
1C. Is there anything else I can help with?
No thank you so much. Bye!

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Entities + Key Phrases + Sentiment
Hello Harry I see you have a flight to Seattle tonight is
that what you’re calling about?
Yes I was hoping I could get a seat upgrade
Sure thing! I see you are a platinum member and there
is a seat in first class number 1C. available Does that
work?
That would be great!
Ok. You are all set and I have upgraded your seat to
1C. Is there anything else I can help with?
No thank you so much. Bye!

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Echo360: Automated Workflow for Efficient Video Captioning
• Echo360 provides leading video platform
technology that helps instructors and
students record, stream, manage and share
interactive video
• The platform captures professor lectures at
650 institutions, in 30 countries, 8,000
classrooms for over 3M students.
• Amazon Transcribe provides high-quality
transcripts for each video, enabling more
powerful search, lower captioning cost, and
enhanced note-taking, making learning
assets more valuable and accessible to
students.

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Easily edit transcripts into Closed Captions
If we we’re doing this in the Southern Hemisphere,
just the opposite.

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
How Do I Get Started?
aws.amazon.com/transcribe

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Translate

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
55%
Of consumers buy only on sites in their own language
Source: CSA Research
Connecting with a Global Audience

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
One option is professional localization
Company specific style, terminology, local dialects, etc.

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
The Problem: Localization Is a Hard Creative Process
Manually intensive Minimal turnaround is
several days
Average cost is $150 per
news article ($0.15 / word)

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
The Opportunity
51% of consumers prefer content in their own language
even if language quality was less than perfect
True in B2B too

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Machine Translation @ Amazon
100s of billions of words
Product listings,
descriptions, search
queries
Website strings
and functional
content
Interpersonal
communication:
customer, vendor and
seller support
Product
documentation and
support content

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Traditional Machine Translation

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Neural Machine Translation

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
A “Pocket” Knife on Amazon.de

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
A Customer Wrote:
Really a very good Pocket knife. Had yesterday after breakfast
something poppy hang between the teeth. Because I once again
found no toothpicks, I ordered mine without further ADO this
pocket knife. The integrated toothpick (Nr. 3) is very stable
and very quickly I could clean my teeth. Super! To the rest I
can tell nothing unfortunately because I him not be emergency.

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
The Amazon Translate Version:
“Really a very good pocket knife. Had a little poppy
between the teeth yesterday after breakfast. Since I didn’t
find a toothpick again, I ordered this pocket knife shortly.
The integrated toothpick (No.3) is very stable and I could
clean my teeth very quickly. Great! I can’t say anything
about the rest because I don’t need it.”

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Use for High-Volume and Time-Sensitive Content
Share of worldwide digital content
Machine translation
• User-authored content
(reviews, forums, etc.)
• Interpersonal
communications (e.g.,
gaming chat, customer
service)
• Insight extraction (e.g.,
sentiment, document
classification)
• Content discovery
• Subtitles / live captioning
• UI
• Marketing materials
• Regulatory filings
• Product documentation
Post-edited
machine
translation

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Features
Translate plain text
Real-time (sentence
< 0.5s on average)
English <> Arabic, Simplified
Chinese, French, German,
Spanish, & Portuguese
Native language
detection
Easy to integrate
Highly secure

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Translate Demo
Real-Time Translation of Interpersonal Communication – Live Chat

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Translate Demo
On-demand Translation of Online Reviews

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
How Do I Get Started?
aws.amazon.com/translate

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Now what?
• Use Amazon Transcribe and Amazon Translate for media and big data
workflows – Join us in the afternoon Workshop: “Automate Video
Captioning & Localization Workflows with ML services from AWS”
• Watch for new patterns on the AWS ML Blog
https://aws.amazon.com/blogs/machine-learning/
• Tell us how we can improve these new services

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Thank you!

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
aws.amazon.com/activate
Everything and Anything Startups
Need to Get Started on AWS