Christian Bandulet The Evolution of Database Technologies

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Christian Bandulet, EMEA Business Development Manager, AWS Database Services
04/02/2019
The Evolution of
Database Technologies

3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Where should I start?

4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
At the very beginning… when evolution started…
CRUD – Create?, Read?, Update?, Delete?
Source: it-times.de
Source:newsnetwork.mayoclinic.org

5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
At the very beginning… when evolution started…
Scale-Up? Scale-Out?
Communication?
Telepathy?
Always available, when you wake up it’s online;
Availability?,
Durability?, SPOF?,
Fault-Tolerance?,
Backup/Restore/DR?
Source: medium.com

6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Moving on…
Insert, Update, Delete?
More like WORM
Scale-Up?
Scale-Out?
Source: deutschlandfunkkultur.de
Source: Wikipedia.org
Source: independentlibraries.co.uk
Long-Term Preservation?
Source: earlyworldhistory.blogspot.com

7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Take it to the next level – Let’s go digital
What if you have many spreadsheets?
Directory
Directory Directory
File
File
File
File
File
File
How to search for content?

8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Database Management System (DBMS)
Instance =
Processes +
Memory +
Logical
Structures
Database =
Physical
Structures
Language - SQL
ISO (ANSI/DIN)
1986
DBMS
DML /DDLCompiler
Query Optimizer
Buffer Pools
Logs
Data Files
Dictionary
Logs
Application
SQL
I/O: e.g. 8KB Blocks (iSCSI, FC, FS, NFS, SMB)

9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Things to consider… Sizing…
Compute:
• CPU GHz
• RAM
I/O:
• Gb/s (Ethernet, iSCSI, FC, SAS, IB, NFS, SMB)
• MB/s (parallel SCSI)
Metrics:
• Bandwidth (theoretical) vs Throughput (actual – i.e. Bandwidth minus overhead)
• IOPS (HDD, SSD, NVMe)
• Latency

10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Demystifying vCPU, Core, CPU, Socket, System
vCPU/Thread
Core
Socket
System/Node
Physical and mechanical
Connection between Printed circuit
Motherboard (PCM) and
microprocessor
CPU aka processor

11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
The Relational Model - E.F. Codd (1970)
Entity Relationship Model (1:1, 1:M, M:M) – 3NF
Optimized for storage!!!
Normalization eliminates redundancy!!!
Source: wikipedia.org
Primary Key/Foreign Key
Complex ad-hoc queries (Joins)

12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Enterprise Information Reference Architecture
Simplified
OLTP
OLTP
OLTP
OLTP
Staging
Ingest Stage Store Report
DWH
Data
Mart
Data
Mart
Data
Mart
Data
Mart
BI
Reporting

13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Data Warehouse (OLAP/ROLAP)
Star Schema
OLAP Cubes
Snowflake Schema
Source: wikipedia.org
Source: wikipedia.org
Source: wikipedia.org
Star schemas center on a central fact
(table) being surrounded by a layer of
dimension tables

14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
HTAP (Hybrid Transactional Analytical Processing)
OLTP+DWH
In-Memory
Columnar
SIMD
OLTP DWH
ELT
ETL
Staging
HTAP
https://hblok.net/blog/posts/2017/12/17/
historical-cost-of-computer-memory-and-storage-4/

15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Use
Data
Produce
Data
The Dark Matter

16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Scaling up a RDBMS…

17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
How to scale out a Relational Database?
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Storage
Application
?

18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
How to scale out a Relational Database?
?
?
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application
Storage Storage
Partition Keys

19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Relational vs Non-Relational (aka NoSQL)
• Optimized for:
• Compute
• Well-known
queries pattern
• Schema-on-Read
• De-normalized
• Optimized for:
• Storage
• Ad-hoc queries
• Schema-on-Write
• Normalized (3NF)

20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
How to scale out a Non-Relational Database?
attribute attribute attribute attribute attribute attribute
attribute attribute attribute
attribute attribute attribute
attribute attribute attribute attribute
attribute attribute attribute attribute
Document Store / Wide-Columnar Store
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application
Storage Storage
Partition Keys
Self-Contained

21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Multi-Modal Database
attribute attribute attribute attribute attribute attribute
attribute attribute attribute
attribute attribute attribute
attribute attribute attribute attribute
attribute attribute attribute attribute
Document Store
Wide-Columnar Store
Document
Graph Store
RDBMS
?
Ledger - Blockchain
SQL
Transactional
ACID

22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Purpose-Built Databases
attribute attribute attribute attribute attribute attribute
attribute attribute attribute
attribute attribute attribute
attribute attribute attribute attribute
attribute attribute attribute attribute
Document Store
Wide-Columnar Store
Graph Store
RDBMS
?
SQL
Transactional
ACID
Ledger - Blockchain

23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
1970 1980 1990 2000
Oracle DB2
SQL Server
MySQL
PostgreSQL
DynamoDB
Redis
MongoDB
Elasticsearch
Neptune
CassandraAccess
Aurora
2010
Timestream
QLDB
DocumentDB

24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
451 Research – Data Platforms Map

25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
What is your database
strategy?
Source: seartec.co.za

26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Agility
Cost Savings
Elasticity
Innovate Faster
Deploy Globally
Re-Hosting (aka Lift &
Shift)
Re-Platform
Re-Factor
IaaS
PaaS
SaaS
Not “if” but “how fast
can we move” and
“what is going to
move first”
Journey to the Cloud

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ENTERPRISE
APPS
DEVELOPMENT & OPERATIONSMOBILE SERVICESAPP SERVICESANALYTICS
Data
Warehousing
Hadoop
/Spark
Streaming
Data Collection
Machine
Learning
Elastic
Search
Virtual
Desktops
Sharing &
Collaboration
Corporate
Email
Backup
Queuing &
Notifications
Workflow
Search
Email
Transcodin
g
One-click App
Deployment
Identity
Sync
Single Integrated
Console
Push
Notifications
DevOps Resource
Management
Application Lifecycle
Management
Containers
Triggers
Resource
Templates
TECHNICAL &
BUSINESS
SUPPORT
Account
Management
Support
Professional
Services
Training &
Certification
Security
& Pricing
Reports
Partner
Ecosystem
Solutions
Architects
MARKETPLACE
Business
Apps
Business
Intelligence
Databases
DevOps
Tools
NetworkingSecurity Storage
Regions
Availability
Zones
Points of
Presence
INFRASTRUCTURE
CORE SERVICES
Compute
VMs, Auto-scaling,
& Load Balancing
Storage
Object, Blocks,
Archival, Import/Export
Databases
Relational, NoSQL,
Caching, Migration
Networking
VPC, DX, DNS
CDN
Access
Control
Identity
Management
Key
Management
& Storage
Monitoring
& Logs
Assessment
and reporting
Resource &
Usage Auditing
SECURITY & COMPLIANCE
Configuration
Compliance
Web application
firewall
HYBRID
ARCHITECTURE
Data
Backups
Integrated
App
Deployments
Direct
Connect
Identity
Federation
Integrated
Resource
Management
Integrated
Networking
API
Gateway
IoT
Rules
Engine
Device
Shadows
Device
SDKs
Registry
Device
Gateway
Streaming
Data Analysis
Business
Intelligence
Mobile
Analytics

28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Modern apps create new requirements
Users: 1M+
Data volume: TB–PB–EB
Locality: Global
Performance: Milliseconds–microseconds
Request Rate: Millions
Access: Mobile, IoT, devices
Scale: Up-out-in
Economics: Pay as you go
Developer access: Instant API accessSocial mediaRide hailing Media streaming Dating

29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Purpose-Built Database Services
DynamoDB NeptuneRDS
Aurora CommercialCommunity
Timestream QLDBElastiCache
Relational Key-value Document In-memory Graph Time series Ledger
DocumentDB
Non-RelationalRelational

30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
If you host your databases on-premises
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
you
App optimization

31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
If you host your databases in Amazon EC2
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
you
App optimization

32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
If you choose a managed Amazon DB service
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
App optimization
High availability
DB s/w installs
OS installation
you
Scaling
Schema design
Query construction
Query optimization

33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Major Trends:
Going Non-Relational:
Going OpenSource:
Moving to the Cloud:

34. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
What will be next?
Gen-Editing with CRISPR/CAS ?
(Clustered Regularly Interspaced Short Palindromic Repeats)
Source: geneticliteracyproject.org

35. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
(GeekWire Chart)
Q4 2018 revenue for the AWS segment
increased 45% y/y to $7.4 billion,
making AWS a nearly $30 billion dollar
run rate business.

36. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Thank you!