No matter if you're thinking of migrating to MongoDB, or need to meet legal requirements for an existing on-prem cluster, this talk has you covered. We start with the basics of replication and sharding and quickly scale up, covering everything you need to know to control your data, and keep it safe from unexpected data loss or downtime - a well-designed MongoDB cluster should have no single point of failure. Learn how others are “stretching” what’s possible but why you shouldn't! I'll present real-world examples from my life in the field in Europe and beyond.

1. Nic Cottrell, MongoDB France
MongoDB cluster design:
from Redundancy to GDPR
@niccottrell
nic-cottrell
niccottrell

2. Who am I?
§ I am currently a Technical Services Engineer.
§ I was recently a Consul;ng Engineer.
§ Before that I was a So>ware Engineer.
§ However, I also completed an MBA.
§ It’s complicated. ! " # $

3. Why are you here?
You know about databases and growth but need to be sure
MongoDB can scale while maintaining data locality and secure
flows.
Your organization has specific requirements that data be on-prem,
encrypted, controlled and backed up with zero-tolerance for
downtime.

4. What will you learn?
By the end of this talk, you will know about:
• Data at scale in multiple physical locations.
• Best (and worst) practices for topology design.
• How to take control of your data geographically.
• Controlling sensitive data in distributed databases.
• Avoiding any single point of failure.
• How to compare on-prem with MongoDB Atlas.

5. Core Principle There should be no single point
of failure in the system.According to me

6. European
Odyssey
1
2
3
4
5
✈
✈

7. Replication and
Sharding
There are two important ways to
scale a MongoDB database:
1. Replicate the same data.
2. Shard (or “partition”) the
data into subsets.
Not the same thing!

8. Replication
London
Paris Frankfurt
PRIMARY
SECONDARY
SECONDARY
PRIMARY
DOWN
Each node stores:
• Data, Indexes
(Encrypted on Enterprise)
• MongoDB logs
New connecJons
AuthenJcaJon aKempts
• Audit logs (Enterprise only)
• FTDC metrics
• System logs

9. Sharding
PRI
SEC SEC
PR I
SEC SEC
PRI
SEC SEC
Shard 1 Shard 2 Shard 3
Application
server
mongos
Config
servers
France Spain Italy

10. Datacenters
and
Geographic
Distribu3on
Let’s talk more about why you
need multiple datacenters

11. Replication and Sharding
London
Paris Frankfurt
PRIMARY
SECONDARY
SECONDARY
Virginia
Ohio California
PRIMARY
SECONDARY
SECONDARY
Sydney
Hong Kong Mumbai
PR IM A RY
SECONDARY
SECONDARY
European Union Data North America Data Asia-Pacific Data
mongos mongos
mongos

12. Features
Defend
Some key features:
• Access Control
• Firewalls, bindIp
• Passwords / x509
• Pseudonymization
• Encryption
• Connections with TLS
• At rest with rotated keys
Detect
• Monitoring and Reporting
redactClientLogData
• Auditing
See more on the blog series: GDPR: Impact to Your Data Management Landscape
New encryption capabilities in MongoDB 4.2:
A deep dive into protecting sensitive workloads
Kenn White, MongoDB - Tuesday, 3pm
Discover
• Compass to explore data
• Automatic data retention with TTL
indexes

13. Regulation (EU)
2016/679
The three Ps:
• permission
• privacy
• protec3on
aka GDPR
The two Ts:
• transfer
• transit
SEEK LEGAL ADVICE
Also HIPAA, PCI-DSS, CCPA
(California),
PIPEDA (Canada) etc.

14. Use cases !
Don’t do any of these!

15. “Mobile data”
the wrong way
!

16. “Mobile” Sharding
PRIMARY
SECONDARY
SECONDARY
PRIMARY
SECONDARY
SEC O N D A RY
PRIMARY
SECONDARY
SECONDARY
North Atlantic Data Mediterranean Data Central Atlantic Data
mongos
❌
Config

17. How’d it do?
• Fast recovery !
• Data has redundant copies
"
• Robust, performant #
• GDPR $
PRIMARY
SECONDARY
SEC O N D A RY
North Atlantic Data

18. What did we learn in the Netherlands?
In order for data to be available and secure:
• Data needs to be in data centers (preferably 3+)
• Large oplogs can be used to let nodes that _do_ go offline to catch
up without a initial sync.
For more details about Mobile databases, check out
§ Hands-on with Realm Mobile Database
Today 1:00pm - 2:45pm
!

19. High Availability
the wrong way
!

20. The setup
• Each node has its own auto-
scaling group, each with a fixed IP
• A Lambda function checks health
and trigger failover
• When a host was considered
failed, it was rebuilt from scratch
• All packages and config were
rebuilt with CloudFormation
• Requires a initial sync each time

21. The setup

22. How’d it do?
!
• Easy to perform post-
mortem
• Fast recovery !
• Data has redundant copies
"
• Robust, performant "
• GDPR #

23. What did we learn in Germany?
• MongoDB is already fault-tolerant.
• With standard topology we can always take down a node for
maintenance, upgrades or debugging.
• The mongodb.log and FTDC data are invaluable to diagnose
crashes or slowdowns.
• We want to avoid initial syncs where ever possible by leaving the
dbPath intact.
• Monitoring and alerting tools are still recommended.
!

24. Multiple NICs
More network cards = Less
points of failure ?
No!
!

25. What did we learn in Italy?
MongoDB is designed for commodity hardware
• Designed to add redundancy and automatic failure for simple,
standard hardware.
• Adding extra complexity from the DevOps side can interfere with
MongoDB health and failover.
• Our documentation addresses many edge cases.
• Reach out to MongoDB Support before you try something
“advanced” in production.
!

26. Best cases !
Please remember this bit!

27. Minimum (viable) topology
eu-west-2a
PR IM A RY
SECONDARY SECONDARY
London
!
eu-west-2b eu-west-2c

28. 2-datacenter topology (on-prem)
Data center 1
PRIMARY
SECONDARY ARBITER
Data center 2 Cloud
Write concern:
Majority
SECONDARY
SECO N D A RY

29. Geo-
Sharding
!
Let’s put it all together:
• Scalability
• Redundancy
• High availability
• Data sovereignty

30. Region-level redundancy
London
Paris Frankfurt
PR IM A RY
SECONDARY
SECONDARY
Virginia
Ohio California
PRIMARY
SECONDARY
SEC O N D A RY
Sydney
Hong Kong Mumbai
PRIMARY
SECONDARY
SECONDARY
European Union North America Asia-Pacific
mongos mongos mongos

31. Example topology
Config servers contain metadata
and shard key values
Each shard contains data for
countries in that region.
With the balancer disabled, no
data is transferred from that
region.
Each web applica>on limits
which country codes it can
process

32. Atlas
! • AWS, Azure, GCP
• Secures your data with individual
VPCs
• Data encrypted with your key
• Best practices for high availability,
fault tolerance
• Automates security upgrades
• MongoDB SOC 2 Security Type II
report available
• Backups all in one region
Fully automated MongoDB
in the Cloud

33. Atlas Global Clusters
• Atlas can automate
regional clusters for
you.
• Focus on
performance (low
latency) not really
compliance right
now.

34. What did we learn?
By now I hope you all agree:
§ For high-availability, we need data in at least three copies of the data,
preferably in separate physical loca>ons.
§ MongoDB provides a good solu>ons to distribute terabytes of data sharded
by workload or geopoli>cal requirements.
§ You can s>ll have a single database, but keep customer data separated.

35. What’s next?
Here’s some other talks that might be interesting:
Tutorial: Hands-on with Realm Mobile Database
§ Alexander Stigsen, Realm/MongoDB
§ Today, 1:00pm - 2:45pm
§ https://sched.co/PULz
Using the New Security Features in MongoDB 4.2
§ Tuesday • 12:45pm - 1:30pm
§ Kevin Albertson, MongoDB
§ https://sched.co/PwAP
New Encryption Capabilities in MongoDB 4.2:
A Deep Dive into Protecting Sensitive Workloads
§ Kenn White, MongoDB
§ Tuesday • 3:00pm - 3:45pm
§ https://sched.co/OJqV

36. Questions?
Reach me at nic.c@mongodb.com
or @niccottrell

37. GDPR Resources

38. GDPR: Frequently Asked Questions
Website
§ How does MongoDB help my organizaAon comply with the GDPR?
§ How does MongoDB Atlas help me comply with the GDPR?
§ What commitments does MongoDB make with respect to the GDPR?
§ hEps://www.mongodb.com/cloud/trust/compliance/gdpr

39. GDPR: Impact to Your Data Management Landscape
Whitepaper
How MongoDB Can Help Meet GDPR Requirements
§ Discover
§ Defend
§ Detect
§ https://webassets.mongodb.com/mongodb_gdpr.pdf

40. GDPR and « The right to be forgotten ».
Checklist
Some key issues to keep in mind:
§ MongoDB Atlas backups can be deleted and re-synced at any time
§ Reference users in a consistent manner to make it easier to find
and delete any historical/log documents by user ID

41. Pseudonymization with MongoDB Views
Blog post
About using Views for access control and auditing
§ https://www.mongodb.com/blog/post/pseudonymization-with-mongodb-
views-the-solution-for-gdpr-and-game-of-thrones-spoilers

42. Extras

43. Audit log examples
Failed authentication
{"atype":"authenticate","ts":{"$date":"2017-02-
14T14:11:29.975+0100"},"local":{"ip":"127.0.1.1","port":27017},"remote":{"ip":"
127.0.0.1","port":42634},"users":[],"roles":[],"param":{"user":"root","db":"adm
in","mechanism":"SCRAM-SHA-1"},"result":18}
Failed insert (due to auth)
{"atype":"authCheck","ts":{"$date":"2017-02-
14T14:15:49.161+0100"},"local":{"ip":"127.0.1.1","port":27017},"remote":{"ip":"
127.0.0.1","port":42636},"users":[{"user":"antun","db":"admin"}],"roles":[{"rol
e":"read","db":"admin"}],"param":{"command":"insert","ns":"test.orders","args":
{"insert":"orders","documents":[{"_id":{"$oid":"58a3030507bd5e3486b1220d"},"id"
:1.0,"item":"paper clips"}],"ordered":true}},"result":13}

44. Redacted log examples
Original log
2017-06-09T13:35:23.446-0400 I COMMAND [conn1] command internal.clients
command: insert {
documents: [ {
_id: ObjectId('593adc5b99001b7d119d0c97'),
name: "Joe", PII: " Sensitive Information"
} ],
...
Redacted log
2017-06-09T13:45:18.599-0400 I COMMAND [conn1] command internal.clients
command: insert {
insert: "###", documents: [ {
_id: "###", name: "###", PII: "###"
} ],
}