Using various NoSQL technologies on AWS, best practices and common pitfalls: What is NoSQL? NoSQL databases overview, sizing considerations, availability and disaster recovery, backup & retention.

1. NoSQL at Scale:
Architecture and Common Pitfalls
Arthur Gimpel
DataZone

2. About Me
• Working with databases for the past 8 years
• 5 years as SQL Server DBA & .NET
• 3 years with NoSQL, Python, Node.js
• 2015 - DataZone

3. DataZone
Data is our business
• High end consultancy & projects
• Private and public training
• Multi vendor MSP (Managed Service Provider)
• Part of Matrix, CloudZone Family
• Proud partners of:

4. NoSQL
Why not?

5. All the wrong reasons..
• Web scale! web scale!
• Cost reduction
• Free school for developers

6. Common Pitfalls & Best
Practices

7. Normalizing in the low cost storage era

8. Expectations gone wild
• Stored procedures
• Jobs
• Triggers
• Mailing Servers
• Shell Commands

9. ODM != ORM
ORM
• Close the modelling
gaps
• Simple Standard
CRUD
ODM
• Typing the untyped
• Retries
• Transactions
• Events

10. Best Practices
• DynamoDB
• Streams: Change Data Capture
• Cross region replication & synchronisation
• Event driven development: Send your new user a
welcome email
• Polyglot Persistence: Initiate an ETL, update your
Elasticsearch index

11. Best Practices
• Elasticsearch
• AWS Plugin:
• Allow service discovery based on security groups
• Run snapshots & restore directly to/from s3
• Storage:
• In small clusters, use EBS.
• In large clusters, use instance store, if you have replication enabled.
• Leverage Watcher & AWS Lambda for anything!

12. Best Practices
• Couchbase
• XDCR:
• Make sure you use cross datacenter replication for multi
region deployments
• Use encrypted communication, especially on XDCR
• Sizing:
• All members must be identical
• Prefer Memory & IO Optimized for most deplyments

13. When Family Can Be Chosen
• Every engine has its flavours - know your tradeoffs
• Splitting drives is good for on prem. EBS is different
• Scaling up & down is always X2
• Prefer linux AMI by AWS- always check for official
support

14. Eventually…
• Don’t work for technology. Make technology for
you. Chose your engines wisely
• Beware of your cluster’s distribution model - don’t
over-estimate elasticity
• Experimenting is easy: Change, Benchmark,
Analyse. Repeat.
• Run engine backups - not EBS snapshots

15. Thank You!
Arthur Gimpel
arthurgi@datazone.io