Modern machine learning workflows leverage AWS services, such as Amazon Transcribe and Amazon Comprehend, to extract, validate, mutate, and enrich your data. Some might drive transactional systems that use machine learning to generate metadata for media assets, while others might derive insights by visualizing customer interaction sentiment from call logs. They all share a common challenge: orchestrating a combination of distinct sequential and parallel steps that are fulfilled by independent microservices. Join us as we examine how workflows can be used to manage that orchestration in a way that is scalable, reliable, and easy to maintain and run. We contrast two approaches for creating such workflows: a traditional monolithic approach and a serverless approach utilizing AWS Steps Functions.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building Serverless
Machine Learning Workflows
Ronald Widha
Mgr. Solutions Architect
Amazon Web Services
A P I 3 1 5

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Who is this Chalk Talk for?
300 Level
ML Scientists
or build your own models using Amazon SageMaker
Data Engineers
You build data processing pipelines, or
automate data-related tasks
AI ML
Serverless Data Processing
Pipeline/Workflow
AI Developers
You use Amazon Lex, Amazon Rekognition, Amazon Comprehend
Chalk Talk
So, ask questions!

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
• Data and machine learning
• Experimentation and continuous learning
• Three workflow patterns

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3
AWS Glue
AWS Data Pipeline
Amazon Kinesis
Streams to lakes

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3
AWS Glue
AWS Data Pipeline
Amazon Kinesis
Pulling the right tools in for analysis
Amazon EC2
Amazon Athena
Amazon EMR

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3
AWS Glue
AWS Data Pipeline
Amazon Kinesis
Leveraging Machine Learning
Amazon EC2
Amazon Athena
Amazon EMR
Amazon
SageMaker
Amazon
Polly
Amazon
Transcribe
Amazon
Comprehend
Amazon
Lex
Amazon
Rekognition
Video
Amazon
Translate
Amazon
Rekognition

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3
AWS Glue
AWS Data Pipeline
Amazon Kinesis
Preparing output for use
Amazon EC2
Amazon Athena
Amazon EMR
Amazon Redshift
Amazon ES
Amazon RDS
Amazon API
Gateway*
Amazon
SageMaker
Amazon
Polly
Amazon
Transcribe
Amazon
Comprehend
Amazon
Lex
Amazon
Rekognition
Amazon
Rekognition
Video
Amazon
Translate

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tens of thousands of customers running machine learning on AWS

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Common patterns #1: Polyglot operational workflow
Ingest Data prep Training Infer Tuning
Machine Learning Workflow
Extract Transform Load Query
Analytics/Materialization Workflow

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingest Data prep Training Infer
Amazon Elastic Compute Cloud
(Amazon EC2)

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
function handle(uri) {
// ingest
const podcastFeed = processPodcastFeed(uri)
// enrich
const vocabName =
createTranscribeVocabulary(podcastFeed)
for (let episode of podcastFeed.episodes) {
const transcript =
transcribe(episode, vocabName)
// mutate
index(transcript)
}
Transcribing podcast feed
Ingest
Enrich
Mutate

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
function handle(uri) {
// ingest
const podcastFeed = await processPodcastFeed(uri)
// enrich
const vocabName =
await createTranscribeVocabulary(podcastFeed)
for (let episode of podcastFeed.episodes) {
const transcript =
await transcribe(episode, vocabName)
// mutate
index(transcript)
}
}
Transcribing podcast feed
Concurrency
control

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
function handle(uri) {
try {
// ingest
const podcastFeed = await processPodcastFeed(uri)
// enrich
const vocabName =
await createTranscribeVocabulary(podcastFeed)
for (let episode of podcastFeed.episodes) {
const transcript =
await transcribe(episode, vocabName)
// mutate
index(transcript)
}
}
catch(e) { … }
}
Transcribing podcast feed – Error handling
Concurrency
control
Error handling

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
function handle(uri) {
try {
// ingest
const podcastFeed = await processPodcastFeed(uri)
// enrich
const vocabName =
await createTranscribeVocabulary(podcastFeed)
for (let episode of podcastFeed.episodes) {
const transcript =
await transcribe(episode, vocabName)
// mutate
index(transcript)
}
}
catch(e) { // if process fail, if transcribe fail }
}
Transcribing podcast feed – Error handling
Concurrency
control
Error handling
retries

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
function handle(uri) {
try {
// ingest
const podcastFeed = await processPodcastFeed(uri)
// enrich
const vocabName =
await createTranscribeVocabulary(podcastFeed)
for (let episode of podcastFeed.episodes) {
const transcript =
await transcribe(episode, vocabName)
// mutate
index(transcript)
}
catch(e) { // if process fail, if transcribe fail }
}
Transcribing podcast feed – State management
Concurrency
control
Operations?
Error handling
retries
State
management?

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
function handle(uri) {
try {
// ingest
const podcastFeed = await processPodcastFeed(uri)
// enrich
const vocabName =
await createTranscribeVocabulary(podcastFeed)
for (let episode of podcastFeed.episodes) {
const transcript =
await transcribe(episode, vocabName)
// mutate
index(transcript)
}
catch(e) { // if process fail, if transcribe fail }
}
Transcribing podcast feed – Distributed systems
Concurrency
control
Operations?
Error handling
retries
State
management?
Process podcast RSS
Create custom vocabulary for Amazon Transcribe
Process podcast episodes

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start
Process podcast RSS
Create custom vocabulary for Amazon Transcribe
Process podcast episodes
End
Start
Start
Start
Start
Process item
Process item
Process item
Process item
AWS Step
Functions

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
// ingest
const podcastFeed = await processPodcastFeed(uri)
// enrich
const vocabName =
await createTranscribeVocabulary(podcastFeed)
for (let episode of podcastFeed.episodes) {
const transcript =
await transcribe(episode, vocabName)
// mutate
index(transcript)
}
}
Common pattern #2:
Monitoring asynchronous processing

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start
Process podcast RSS
Create custom vocabulary for Amazon Transcribe
Process podcast episodes
End
Start
Start
Start
Start
Process item
Process item
Process item
Process item
Is vocabulary completed?
Wait for vocabulary creation
Check vocabulary status
Wait 30 seconds
Are all episodes processed?
AWS Step
Functions

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start
Process podcast RSS
Create custom vocabulary for Amazon Transcribe
Process podcast episodes
End
Start
Start
Start
Start
Process item
Process item
Process item
Process item
Is vocabulary completed?
Wait for vocabulary creation
Check vocabulary status
Wait 30 seconds
Are all episodes processed?
AWS Step
Functions

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Common pattern #3: Data available at all stages
Ingest Data prep Training Infer

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start
Process podcast RSS
Create custom vocabulary for Amazon Transcribe
Process podcast episodes
End
Start
Start
Start
Start
Process item
Process item
Process item
Process item
Is vocabulary completed?
Wait for vocabulary creation
Check vocabulary status
Wait 30 seconds
Are all episodes processed?
AWS Step
Functions

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start
Process podcast RSS
Create custom vocabulary for Amazon Transcribe
Process podcast episodes
End
Start
Start
Start
Start
Process item
Process item
Process item
Process item
Is vocabulary completed?
Wait for vocabulary creation
Check vocabulary status
Wait 30 seconds
Are all episodes processed?
Common pattern #4:
Child workflows

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ingest enrich mutate use
AWS Step
Functions
AWS Glue Amazon
SageMaker
Common pattern #4: Child workflows

28. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ronald Widha
@ronaldwidha

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS product and resource icons
(Updates coming August 2018)
Download icons to use in
your presentation here:
https://aws.amazon.com/
architecture/icons/