AWS Data-Driven Insights Learning Series_ANZ Sep 2019 Part 2

© 2019, Amazon Web Services, Inc. or its Affiliates.
Tom McMeekin
Solutions Architect
AWS
The Future of Cloud Data
Warehousing

–

every 5
years
15
years
live for
1,000x
scale
>10x
grows
*IDC, Data Age 20215: The Evolution of Data to Life-Critical Don’t Focus on Big Data, Focus on the Data That’s Big, April 2017.
How do I provide democratized
access to data to enable
informed decisions while at the
same time enforce data
governance and prevent
mismanagement of the data?
Hadoop Elasticsearch Presto Spark
Democratization
of data
Governance
& control
Organic revenue growth
*Aberdeen: Angling for Insight in Today’s Data Lake, Michael Lock, SVP Analytics and Business Intelligence
more valuable
11 8 5 4

Democratization
of data
Governance
& control
Hadoop Elasticsearch Presto Spark
Apache
Parquet

New use cases are emerging for the modern Data Warehouse
clickstream
improve ad targeting
predict
risk analysis, fraud detection
Aggregate transactional
in-game
behavior
personalized experiences

/
/
/
/
Broken view of your business and your customers

process more than 2 Exabytes of data
Most popular Fastest

Based on the cloud DW benchmark derived from TPS-DS 30 TB dataset, 4-node cluster
Redshift Vendor 1 Vendor 2
Queries Per Hour
(Higher is better)

For every 24 hours your main
cluster is in use, we’ll provide a
one-hour credit for concurrent
cluster usage
97% of users will never see a
charge for auto-scale resources
0
2000
4000
6000
8000
10000
12000
5 40 80 120 150 180
QueriesperHour
(QpH)
Number of concurrently active users
Throughput scales linearly
Amazon Redshift’s throughput scales linearly
with concurrent users

Amazon Redshift Concurrency Scaling
Backup
Redshift automatically adds transient clusters,
in seconds, to serve sudden spike in concurrent requests with
consistently fast performance. No hydration required.
Caching Layer
How it works:
All queries go to the leader
node, user only sees less wait
for queries.
When queries in designated
WLM queue begin queuing,
Redshift automatically routes
them to the new clusters,
enabling Concurrency Scaling
automatically.
Redshift automatically spins up a
new cluster, processes waiting
queries and automatically shuts
down the Concurrency Scaling
cluster.
1
2
3
For every 24 hours that your
main cluster is in use, you
accrue a one-hour credit for
Concurrency Scaling. This
means that Concurrency
Scaling is free for > 97% of
customers.

Enabling Concurrency scaling

to Redshift cluster in busy
periods
transition time
compute
and storage
on-demand
Scale up and down in minutes
Redshift
Cluster
Compute
nodes
Redshift Managed S3
JDBC/ODBC
Leader Node
CN2CN1 CN3 CN4
Backup

Fastest Most cost-effective
up to 75% $758,845
average annual benefits
per TB per year
$319,300
higher revenue
per 100TB per year
469%
Less than the #2 cloud DW with
on-demand pricing and 75% less
with Reserved Instances (RIs)
*based on IDC’s “ROI of Amazon Redshift paper”, 2017

Fastest Most cost-effective Integrates with your data lake

Security is built-in
Select compliance certifications*
10 GigE (HPC)
SQL Clients/BI Tools
Customer
VPC
Internal
VPC
JDBC/ODBC
Compute
Nodes
Leader
Node
Network isolation
End-to-end encryption
Integration with AWS Key
Management Service
Amazon S3
Fastest Most cost-effective Integrates with your data lake

Unicorn Nation
UNICORN NATION
UnicornNation is a global entertainment company that
provides ticketing, merchandising and promotion of large
concerts and events.
In recent years, they have been collecting data through a
number of disparate systems and have consolidated this
data onto a modern data architecture
This includes a large volume of data stored in the Unicorn
Nation data lake, including both structured and
unstructured data used by a wide range of users. In
addition to a data lake, they have also created a data
warehouse, which they use for dashboards and reports.

Unicorn Nation

© 2019, Amazon Web Services, Inc. or its Affiliates.© 2019, Amazon Web Services, Inc. or its Affiliates.
Demo

—

Visit the Redshift website
Watch the Redshift sessions
from re:Invent 2018 featuring
Dow Jones, McDonald’s,
Intuit, and Equinox Fitness
Read the DW Modernization
e-book
Start free with the two-
month trial
Set up a POC with the
guidebook
Self-service: Migrate to
Redshift using Data Migration
Service and Schema
Conversion tool
Work with a partner

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you
Tom McMeekin
tommcm@amazon.com

Eric Greene
AI & ML Solutions Specialist
AWS
AI & ML and Data Lakes: A
platform to build business
outcomes from data

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data lakes enable analytics and machine learning
Cost-effective
Scalable and durable
Secure
Open and comprehensiveAnalyticsMachine learning
Real-time data
movement
On-premises
data movement
Data lake
on AWS

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its Affiliates.
What is
successful
machine learning?

40% of digital transformation initiatives
supported by AI in 2019
—IDC 2018
InnovationDecision
making
Customer
experience
C E N T E R P I E C E F O R D I G I TA L T R A N S F O R M AT I O N
Business
operations
Competitive
advantage

Our mission at AWS
Put machine learning in the
hands of every developer

200 new features and services
launched this last year alone
Unmatched flexibility
Broadest and
deepest set of AI
and ML services
70% cost reduction
in data-labeling
10x faster performance
75% lower inference cost
Accelerate your
adoption of ML
with SageMaker
Built on the most
comprehensive cloud
platform
AWS Named as a Leader in
Gartner’s Infrastructure as a Service
(IaaS) Magic Quadrant for the 9th
Consecutive Year
W H Y A W S F O R M L ?

AI Services
Broadest and deepest set of capabilities
T H E A W S M L S T A C K
VISION SPEECH LANGUAGE CHATBOTS FORECASTING RECOMMENDATIONS
ML Services
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D
& C O M P R E H E N D
M E D I C A L
L E X F O R E C A S TR E K O G N I T I O N
I M A G E
R E K O G N I T I O N
V I D E O
T E X T R A C T P E R S O N A L I Z E
Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment HostingAmazon SageMaker
FRAMEWORKS INTERFACES INFRASTRUCTURE
ML Frameworks + Infrastructure
F P G A SE C 2 P 3
& P 3 D N
E C 2 G 4
E C 2 C 5
I N F E R E N T I AG R E E N G R A S S E L A S T I C
I N F E R E N C E
D L
C O N T A I N E R S
& A M I s
E L A S T I C
K U B E R N E T E S
S E R V I C E
E L A S T I C
C O N T A I N E R
S E R V I C E

Modernize your contact center to improve customer service
P U T M L TO W O R K F O R Y O U R B U S I N E S S
conversational chat bots | call transcription | intelligent routing | sentiment analysis
VoC analytics text-to speech | multilingual omni-channel communication
POLLY TRANSCRIBE TRANSLATE COMPREHEND LEX

VOC analytics for customer service
Amazon
S3
Amazon
Athena
Amazon
Translate
Amazon
Comprehend
Amazon
Transcribe
sentiment, entities,
and key phrases
Call
Recordings
Transcribe and analyze historical call
recordings to measure customer
sentiment over time
• specific topics
• products and issues

Live call analysis with next best action and
translation

Use AI services to strengthen safety and security
accurate facial analysis | identity protection | metadata extraction
REKOGNITION
IMAGE
COMPREHEND &
COMPREHEND
MEDICAL
REKOGNITION
VIDEO

Fighting human
trafficking
Marinus Analytics uses Amazon Rekognition to find
human trafficking victims, then help law enforcement
prosecute the traffickers.

Automate media workflows to reduce costs and monetize content
content moderation | contextual ad insertion | searchable media library
custom facial recognition | multi-language metadata search
REKOGNITION
IMAGE
REKOGNITION
VIDEO
COMPREHEND TRANSCRIBE TRANSLATE TEXTRACT

Monitoring radio & TV
content
“At Isentia, we enable customers to analyze and monitor
the media coverage for their brands. We create more than
13K summaries per day from radio and TV content. With
Amazon Transcribe, we can transcribe all the audio/video
content that we monitor and analyze the text data with
Amazon Comprehend. Features like timestamps and
punctuation make it very easy for us to search through the
data and drill down and present key insights for our
customers to review.”
Andrea Walsh - CIO, Isentia

Reduce localization costs and improve accuracy
custom vocabulary | timestamp generation | secure real-time translation | language identification
POLLY TRANSCRIBE TRANSLATE COMPREHEND

Scaling real-time
translation
Using Amazon Translate, Lionbridge is able to
scale machine translation in order to localize
content faster and in more languages. Using
Translate, Lionbridge was able to reduce
translation costs by 20 percent.

Accurately forecast future business outcomes
forecasting technology used by Amazon.com | multiple time-series data
forecast scheduling and visualization | supply chain integration
FORECAST

Personalize customer experiences with targeted recommendations
recommendation technology used by Amazon.com | context-aware recommendations
sentiment analysis | VoC analytics
PERSONALIZE REKOGNITION
IMAGE
REKOGNITION
VIDEO
COMPREHEND

Personalizing
customer experiences
Domino’s uses Amazon Personalize to customize
and scale relevant marketing communications to
customers based on time, context, and content,
thereby improving and enhancing their experience
with the Domino’s brand.

Increase efficiency with automated document analysis
optical character recognition (OCR) | automatic data extraction | natural language processing
intelligent search | text analytics
TEXTRACT COMPREHEND &
COMPREHEND
MEDICAL

AI Services
T H E AW S M L S TA C K
ML Services
M E D I C A L
I M A G E
V I D E O
F P G A SE C 2 P 3
& P 3 D N
E C 2 G 4
E C 2 C 5
I N F E R E N C E
D L
C O N T A I N E R S
& A M I s
E L A S T I C
K U B E R N E T E S
S E R V I C E
E L A S T I C
C O N T A I N E R
S E R V I C E

Smart statistics with
machine learning
Now on AWS, Formula 1 will extract critical
performance statistics to make race predictions
and deliver a next level viewing experience to its
fans. With 65 years of historical race data, stored
in Amazon S3, Formula 1 data scientists will train
deep learning models using Amazon
SageMaker to power F1 Insights. AWS will give
Formula 1 the tools necessary to capture and
deliver data at unmatched accuracy and speed.

The path to ML-driven insights and predictions
While the power of ML is unrivaled,
“data scientists spend around 80% of
their time on preparing and
managing data for analysis” … hence
only 20% of their time is used to
derive insights
6–18
months
Fetch
data
Clean &
format
data
Prepare &
transform
data
Evaluate
model
Integrate
with prod
Monitor/
debug/
refresh
Train
model

Bringing machine learning to all developers
A M A Z O N S A G E M A K E R
Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems

Successful models require high-quality data

Amazon SageMaker Ground Truth
How it works

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
• K-Means Clustering
• Principal Component Analysis
• Neural Topic Modelling
• Factorization Machines
• Linear Learner (Regression)
• BlazingText
• Reinforcement learning
• XGBoost
• Topic Modeling (LDA)
• Image Classification
• Seq2Seq
• Linear Learner (Classification)
• DeepAR Forecasting

Over 200 algorithms and models
Natural Language
Processing
Grammar & Parsing Text OCR Computer Vision
Named Entity
Recognition
Video Classification
Speech Recognition Text-to-Speech Speaker Identification Text Classification 3D Images Anomaly Detection
Text Generation Object Detection Regression Text Clustering
Handwriting
Recognition
Ranking
A V A I L A B L E A L G O R I T H M S & M O D E L S
S E L E C T E D V E N D O R S

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training Optimization

Amazon SageMaker Neo: Train once, run anywhere
Neo

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
One-click
deployment

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
One-click
deployment
Fully managed with
auto-scaling, health checks,
automatic handling
of node failures,
and security checks

Amazon SageMaker demo

AI Services
T H E A W S M L S T A C K
ML Services
M E D I C A L
I M A G E
V I D E O
F P G A SE C 2 P 3
& P 3 D N
E C 2 G 4
E C 2 C 5
I N F E R E N C E
D L
C O N T A I N E R S
& A M I s
E L A S T I C
K U B E R N E T E S
S E R V I C E
E L A S T I C
C O N T A I N E R
S E R V I C E

Introducing Reinforcement learning (RL)
Reinforcement learning
(RL)
Supervised learning
(ASR, computer vision)
Unsupervised learning
(Anomaly detection,
identifying text topics)
Amount of labeled training data required
Complexityofdecisions

Amazon SageMaker RL
Reinforcement learning for every developer and data scientist
Broad support
for frameworks
Broad support for simulation
environments
2D & 3D physics
environments and
OpenGym support
Support Amazon Sumerian,
AWS RoboMaker and the open
source Robotics Operating
System (ROS) project
Fully
managed
Example notebooks
and tutorials
K E Y F E A T U R E S

• Build machine learning models
in Amazon SageMaker
• Train, test, and iterate on the track using
the AWS DeepRacer 3D racing simulator
• Compete in the world’s first global autonomous
racing league, to race for prizes and a chance to
advance to win the coveted AWS DeepRacer Cup
A fully autonomous 1/18th-scale race car designed to help you
learn about reinforcement learning through autonomous driving
AW S D E E P R A C E R

The world’s first deep learning-
enabled video camera for developers
AW S D E E P L E N S
• Purpose-built for ML-skills development
• Fully programmable and customizable
• Build custom Amazon SageMaker models
• 10-minutes to your first deep learning project

H O W W E C A N H E L P
• Brainstorming
• Custom modeling
• Training
• Work side-by-side with Amazon experts
ML Solutions Lab
• Practical education on ML for new
and experienced practitioners
• Based on the same material used
to train Amazon developers
Machine Learning
Training and Certification

ml.aws
Thank you!

AWS Data-Driven Insights Learning Series_ANZ Sep 2019 Part 2

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie AWS Data-Driven Insights Learning Series_ANZ Sep 2019 Part 2

Ähnlich wie AWS Data-Driven Insights Learning Series_ANZ Sep 2019 Part 2 (20)

Mehr von Amazon Web Services

Mehr von Amazon Web Services (20)

AWS Data-Driven Insights Learning Series_ANZ Sep 2019 Part 2