Speaker: Julio Viera, VP Engineering - Backend as a Service, Fuze Level: 300 (Advanced) Track: Application Architecture At Fuze, we are going through the process of re-architecting our backends and we decided to use a microservices approach. Microservice architectures face common challenges like geo-distribution of data, retention periods, and security rules. We found that MongoDB with zone sharding enabled us to address these concerns effectively. We created a service called The Floppy, which is a RESTful Object Storage that automatically scales and distributes the data around the world. The Floppy also supports real-time queries via WebSockets and advanced security rules using expressions that are evaluated in real time. What You Will Learn: - How to deploy MongoDB globally with Zone Sharding (also known as Tag Aware Sharding). - How to abstract application logic from the Zone Sharding architecture. - How to implement a publish and subscribe framework that evaluates document writes and triggers events to applications listening for changes.

1. Globally Distributed
RESTful Object Storage
MongoDB World 2017

2. Who am I?
What is Fuze?

3. For today's presentation
Requirements 5min
Architecture 5min
Schema 5min
PubSub 5min
Sharding 10min
Binary storage
Query Language
Questions 5min

4. The requirements

5. The requirements
Document and binaries
Scalable / High available
Support retention
Authentication
Backup
Data change notifications
Accessible from clients

6. introducing...
the floppy
* not limited by 1.44MB

7. Stats
5 teams in the company using Floppy
Chat attachments, retention info and sessions
User settings for some deskphones
Internal beta:
Meetings metadata storage/sync
Next iteration of chat and presence
2.6 MM operations per day: ~30/sec 1000/sec peak
45 Collections
50 MM Documents

8. Architecture

9. Why Mongo
+5 years using it
Document storage
Developer friendly
Designed for high availability and scalability
Natively supports sharding and zone sharding
Natively supports TTL indexes

10. Why RESTful
Abstraction layer
Rate limit
Control Query complexity via Query Language
Control the operations done in the DB
Accessed over the internet - clients can use it as a DB in the cloud
We can reuse our current authentication system

11. Services
Services
Floppy
Mongo
cluster
Services https
https
Private to Floppy
Internal corporate
network
Public internet
High Level Architecture

12. Secondary Secondary
Primary
Config
servers
Secondary Secondary
Primary
Shard 0
Secondary Secondary
Primary
Shard N
...
MongoDB
3.4
Wiredtiger

13. Stack
Java
RestEASY / Netty
MongoDB Java Driver
Oplog consumer based on two popular open source libraries
AWS
EC2 for MongoDB
ElasticBeanstalk & Docker for the App

14. Schema
/users/123/chat/conversations/X/messages/Y
/users/123/voicemails
/tenants/cia/confidential_archives/7328183

15. Schema - Floppy structures
Storage
Collections
Sub-collections
Documents

16. Schema - Url to Floppy structures
Storage
Collections
Sub-collections
Documents
/users/123/chat/conversations/X/messages/Y

17. Schema - Url to Floppy structures
Storage: chat
Collection: conversations
Sub-collection: messages
Documents: conv or message
/users/123/chat/conversations/X/messages/Y

18. Schema - Url to Mongo structures
Storage: storage_chat
(maybe to db in the future)
Collection: chat_conversations
Sub-collection: chat_conversations_messages
Documents: mongo documents
(messages have a parent id)
/users/123/chat/conversations/X/messages/Y

19. PubSub
Change notifications

20. Rest
Interface
Data
Adapter
Service X
Insert
document
id = 123
Collection = users
MongoS
Secondary
Secondary
Primary
MongoDB Shard 0
Oplog
https
PubSub
Change notifications
START

21. Rest
Interface
Data
Adapter
Service X
Insert
document
id = 123
Collection = users
MongoS
Secondary
Secondary
Primary
MongoDB Shard 0
Oplog
Oplog
Consumers
Notification
Service
Location change:
document_insert
/users/
/users/123
https
https
PubSub
Change notifications
START
END
Subscribers

22. Sharding

23. Our Sharding philosophy
We always design with sharding in mind
We always start with sharded deployments (1 shard)
We always have at least 3 replica set members per shard

24. Goals
User's data should live in a single chunk until it is outgrown
Tenant data should live in a single chunk until it is outgrown
Sharding complexity must be hidden from the REST interface

25. Sharding Key
{
loc: 0
tenant: "CIA",
user: 123,
_id: 981
...
}

26. Sharding Key - hiding complexity
/users/123/chat/conversations/X
RESTful URL:
{
loc: 0
tenant: "CIA",
user: 123
_id: "X",
payload:{
name: "some conversation",
participants: 5
}
}
Conversations collection:

27. {
loc: 0
tenant: "CIA",
user: 123
_id: "X",
payload:{
name: "some conversation",
participants: 5
}
}
Sharding Key - hiding complexity
/users/123/chat/conversations/X
Conversations collection:
user to
tenant
tenant
to
loc
RESTful URL:

28. Sharding Example

29. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"

30. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1

31. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2

32. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3

33. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3
_id: 4

34. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3
_id: 4
Split!

35. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3
_id: 4

36. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4Move
Chunks!

37. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5

38. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5 _id: 6

39. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5 _id: 6
Split!

40. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5 _id: 6

41. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5 _id: 6
_id: 7

42. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5 _id: 6
_id: 7
New shard
in cluster

43. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4 _id: 5 _id: 6
_id: 7
Move
Chunks!

44. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5 _id: 6
_id: 7
User's data should live in a single chunk until
it is outgrown

45. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 5 _id: 6
_id: 7
_id: 3 _id: 4
Location: 0
Tenant: "NASA"
User: "namstrong"
Location: 0
Tenant: "NASA"
User: "baldrin"

46. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 5 _id: 6
_id: 7
_id: 3 _id: 4
_id: 10
Location: 0
Tenant: "NASA"
User: "namstrong"
Location: 0
Tenant: "NASA"
User: "baldrin"

47. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5 _id: 6
_id: 7_id: 10
_id: 11
Location: 0
Tenant: "NASA"
User: "namstrong"
Location: 0
Tenant: "NASA"
User: "baldrin"

48. Sharding Example Location: 0
Tenant: "CIA"
User: "jsmith"
_id: 1
_id: 2
_id: 3 _id: 4
_id: 5 _id: 6
_id: 7
Location: 0
Tenant: "CIA"
User: "jsmith"
Tenant data should live in a single chunk
until it is outgrown
_id: 10
_id: 11
Location: 0
Tenant: "NASA"
User: "namstrong"

49. Thanks!

50. questions?