How do you design a high performance and scalable database for a global e-commerce application or build a datastore to power a high volume recommendations engine or ingest and query millions of timeseries events from IoT sensors? While relational databases have proven to be successful in the past, emergence of newer application use cases create a need for specialised data persistence and query models. In this session, you will learn about use cases where non-relational databases can be a better choice and explore the suite of AWS non-relational data stores.

1. S U M M I T
SYDNEY

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Right tool for the right job:
journey beyond relational databases
Syed Jaffry
Solutions Architect
Amazon Web Services

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Common data categories and use cases
Key-value
Real-time
bidding,
shopping cart,
social, product
catalog,
customer
preferences
Amazon
DynamoDB
Document
Content
management,
personalisation,
mobile
Amazon
DocumentDB
(with MongoDB
compatibility)
In-memory
Leaderboards,
real-time
analytics,
caching
Amazon
ElastiCache
Graph
Fraud detection,
social
networking,
recommendation
engine
Amazon
Neptune
Ledger
Systems
of record, supply
chain, health care,
registrations,
financial
Amazon
Quantum Ledger
Database (QLDB)
Time-series
IoT applications,
event tracking
Amazon
Timestream
RDBMS
Lift and shift, ERP,
CRM, finance
Amazon
RDS

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Why do I need to think about different
databases?
Why can’t I model all of my use cases in
just one database?

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Modern apps create new requirements
Users: 1 million+
Locality: Global
Devices: Smartphones, Wearables, IoT
User Interfaces: Web, mobile, voice, chatbots
Data volume: TB–PB
Performance: Milliseconds–microseconds
Scale: Up-down, Out-in
Economics: Pay for what you use
Developer access: No assembly requiredSocial mediaRide hailing Media streaming Dating

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Changing nature of user interactions
”The future of user experiences” - Gartner report October 2018

7. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Breaking down big applications into smaller ones

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Breaking down big applications into smaller ones

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Airbnb uses different databases based
on the purpose
User search history: Amazon DynamoDB
• Massive data volume
• Need quick lookups for personalised search
Session state: Amazon ElastiCache
• In-memory store for submillisecond site rendering
Relational data: Amazon RDS
• Referential integrity
• Primary transactional database

11. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Key-value data
• Simple key value
pairs
• Partitioned by keys
• Resilient to failure
• High throughput,
low-latency reads
and writes
• Consistent
performance at
scale
Gamers
Primary Key Attributes
GamerTag Level Points High Score Plays
Hammer57 21 4050 483610 1722
FluffyDuffy 5 1123 10863 43
Lol777313 14 3075 380500 1307
Jam22Jam 20 3986 478658 1694
ButterZZ_55 7 1530 12547 66
… … … … …
PUT {
TableName:"Gamers
Item: {
"GamerTag":"Hammer57
"Level":21
"Points":4050,
"Score":483610,
"Plays":1722
} }
GET {
TableName:"Gamers",
Key: {
"GamerTag":"Hammer57“,
“ProjectionExpression“:”Points”
} }

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Gamers
Primary Key
Attributes
Gamer Tag Type
Hammer57
Rank
Level Points Tier
87 4050 Elite
Status
Health Progress
90 30
Weapon
Class Damage Range
Taser 87% 50
FluffyDuffy
Rank
Level Points Tier
5 1072 Trainee
Status
Health Progress
37 8
Key-value use case
// Status of Hammer57
GET {
TableName:"Gamers",
Key: {
"GamerTag":"Hammer57",
"Type":"Status” } }

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Gamers
Primary Key
Attributes
Gamer Tag Type
Hammer57
Rank
Level Points Tier
87 4050 Elite
Status
Health Progress
90 30
Weapon
Class Damage Range
Taser 87% 50
FluffyDuffy
Rank
Level Points Tier
5 1072 Trainee
Status
Health Progress
37 8
Key-value use case
// Status of Hammer57
GET {
TableName:"Gamers",
Key: {
"GamerTag":"Hammer57",
"Type":"Status” } }
// Return all Hammer57
TableName:“Gamers”,
KeyConditionExpression:"GamerTag = :a
ExpressionAttributeValues: {
":a Hammer57” } }

16. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Graph data
• Relationships are first-class
objects
• Vertices connected by Edges
PURCHASED PURCHASED
FOLLOWS
PURCHASED
KNOWS
PRODUCT
SPORT
FOLLOWS

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Graph use case
Product recommendation to a user
PURCHASED PURCHASED
FOLLOWS
PURCHASED
KNOWS
PRODUCT
SPORT
FOLLOWS
FOLLOWS

19. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Document data
• Data is stored in JSON-like
documents
• Documents map naturally to
how humans model data
• Flexible schema and indexing
• Expressive query language
built for documents (ad hoc
queries and aggregations)
{
id: 1,
name: "sue",
age: 26,
email: "sue@example.com",
promotions: ["new user", "5%", "dog
lover"],
memberDate: 2018-2-22,
shoppingCart: [
{product:"abc", quantity:2,
cost:19.99},
{product:"edf", quantity:3, cost:
2.99}
]
}
Documents are first-class objects
in the database

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Document data
• Data is stored in JSON-like
documents
• Documents map naturally to
how humans model data
• Flexible schema and indexing
• Expressive query language
built for documents (ad hoc
queries and aggregations)
{
id: 1,
name: "sue",
age: 26,
email: "sue@example.com",
promotions: ["new user", "5%", "dog
lover"],
memberDate: 2018-2-22,
shoppingCart: [
{product:"abc", quantity:2,
cost:19.99},
{product:"edf", quantity:3, cost:
2.99}
]
}
Documents are first-class objects
in the database

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Document use case
MobileContent
management
Personalisation
Catalog User profilesRetail and
marketing

23. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Record keeping today
Banking and finance
Keeping track of transactions,
trades and accounts
Manufacturing
Recording components used
in manufacturing
Transport and logistics
Tracking transportation
of goods
E-Commerce
Where’s my stuff?
HR and payroll
Tracking changes to an
individual’s profile
Government
Tracking vehicle title
history

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Challenges with building ledgers
Adds
unnecessary
complexity
BlockchainRDBMS - audit tables
Difficult to
maintain
Hard to use
and slow
Hard to
build
Custom audit functionality using
triggers or stored procedures
Impossible
to verify
No way to verify changes made
to data by sys admins

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Quantum Ledger Database (QLDB) (Preview)
Fully managed ledger database
Track and verify history of all changes made to your application’s data
Immutable
Maintains a sequenced record of
all changes to your data, which
cannot be deleted or modified;
you have the ability to query and
analyse the full history
Cryptographically
verifiable
Uses cryptography to
generate a secure output
file of your data’s history
Easy to use
Easy to use, letting you
use familiar database
capabilities like SQL APIs
for querying the data
Highly scalable
Executes 2–3X as many
transactions than ledgers
in common blockchain
frameworks

27. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Time-series data
What is time-series data?
A sequence of data points
recorded over a time interval

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Time-series data
What is time-series data?
t

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Time-series data
What is time-series data?
t

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Time-series data
What is time-series data?
What is special about a
time-series database?
Time is the
single primary axis
of the data model
t

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Time-series use case
Application events
IoT Sensor
Readings
DevOps data
Humidity
% WATER VAPOR
91.094.086.093.0

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Timestream (sign up for the preview)
Fast, scalable, fully managed time-series database
1,000x faster and 1/10th the
cost of relational databases
Collect data at the rate of
millions of inserts per
second (10M/second)
Trillions of
daily events
Adaptive query processing
engine maintains steady,
predictable performance
Time-series analytics
Built-in functions for
interpolation, smoothing,
and approximation
Serverless
Automated setup,
configuration, server
provisioning, software patching

34. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Retail demo application
Demo application:
1. Available today
2. On GitHub:
/aws-samples/aws-
bookstore-demo-app
3. One click
CloudFormation
deployment
Search
Indexing and
searching
semistructured
logs and data
Product
search
Amazon
Neptune
Amazon Elasticsearch
Service
Key-value
High
throughput,
Low-latency
reads
and writes,
endless scale
Shopping cart, user
profile
Graph
Quickly and
easily create
and navigate
relationships
between
data
Product
recommendation
In-memory
Query by key
with
microsecond
latency
Product
leaderboard
DynamoDB ElastiCache

36. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
CHALLENGE
Wanted to enable anyone to learn a
language for free.
SOLUTION
Purpose-built databases from AWS:
• DynamoDB: 31B items tracking
which language exercises completed
• Aurora: primary transactional
database for user data
• ElastiCache: instant access to
common words and phrases
Result:
More people learning a language on
Duolingo than entire US school system
300M total users
7B exercises per month

38. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Purpose-built databases
Relational Key-value Document In-memory Graph Time-series Ledger
Amazon
DynamoDB
Amazon
Neptune
Amazon
RDS
Amazon
Timestream
Amazon
QLDB
Amazon
ElastiCache
Amazon
DocumentDB

40. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Syed Jaffry
syejaffr@amazon.com