1. MongoSF
April 30, 2010
1

2. Hot Potato Overview
• What: Hot Potato connects audiences around shared interests
• When: Founded early 2009 / PoC: Dec 2009 / Beta: 2 months ago
• Where: Williamsburg, Brooklyn!
• Who: Lincoln, Justin, Saadiq, Jeremy, Jace, Matt, Mike, Will
2

3. hotpotato.com
3

4. Hot Potato on the iPhone
4

5. Technology Stack
Design principles: Simple and stateless
API
• Scala / MongoDB
API Clients
• Web application (Python/Django)
• iPhone (Obj-C)
5

6. Hot Potato API - Why MongoDB?
• Good documentation and excellent support
• Fully Featured (like an RDBMS) but with a migration path to scale-
out
• Replication
• Easy administration / scripting
• Fast
• Auto-sharding coming soon
6

7. Hot Potato API - Why MongoDB?
7

8. Hot Potato API - Why MongoDB?
RDBMS Optimization process
7

9. Hot Potato API - Why MongoDB?
RDBMS Optimization process
1. too many reads -> add cache
7

10. Hot Potato API - Why MongoDB?
RDBMS Optimization process
1. too many reads -> add cache
2. too many joins -> de-normalize your data
7

11. Hot Potato API - Why MongoDB?
RDBMS Optimization process
1. too many reads -> add cache
2. too many joins -> de-normalize your data
3. too many writes -> scale up hardware, CPU, RAM, IO
7

12. Hot Potato API - Why MongoDB?
RDBMS Optimization process
1. too many reads -> add cache
2. too many joins -> de-normalize your data
3. too many writes -> scale up hardware, CPU, RAM, IO
4. too much db load -> eliminate triggers, stored procedures
7

13. Hot Potato API - Why MongoDB?
RDBMS Optimization process
1. too many reads -> add cache
2. too many joins -> de-normalize your data
3. too many writes -> scale up hardware, CPU, RAM, IO
4. too much db load -> eliminate triggers, stored procedures
5. too much db load -> pre-materialize complex queries
7

14. Hot Potato API - Why MongoDB?
RDBMS Optimization process
1. too many reads -> add cache
2. too many joins -> de-normalize your data
3. too many writes -> scale up hardware, CPU, RAM, IO
4. too much db load -> eliminate triggers, stored procedures
5. too much db load -> pre-materialize complex queries
6. writes bottleneck -> drop secondary indexes
7

15. Hot Potato API - Why MongoDB?
RDBMS Optimization process
1. too many reads -> add cache
2. too many joins -> de-normalize your data
3. too many writes -> scale up hardware, CPU, RAM, IO
4. too much db load -> eliminate triggers, stored procedures
5. too much db load -> pre-materialize complex queries
6. writes bottleneck -> drop secondary indexes
At this point you are using the RDBMS as a KV store. The full-
featured nature of the RDBMS is merely getting in the way of scaling.
7

16. Hot Potato API - Why MongoDB?
8

17. Hot Potato API - Why MongoDB?
Mongo comparison
8

18. Hot Potato API - Why MongoDB?
Mongo comparison
1. reads start out fast, can be externally cached, can also be
horizontally scaled
8

19. Hot Potato API - Why MongoDB?
Mongo comparison
1. reads start out fast, can be externally cached, can also be
horizontally scaled
2. mongo doesn’t support joins - data starts de-normalized
8

20. Hot Potato API - Why MongoDB?
Mongo comparison
1. reads start out fast, can be externally cached, can also be
horizontally scaled
2. mongo doesn’t support joins - data starts de-normalized
3. writes start out fast, can be horizontally scaled
8

21. Hot Potato API - Why MongoDB?
Mongo comparison
1. reads start out fast, can be externally cached, can also be
horizontally scaled
2. mongo doesn’t support joins - data starts de-normalized
3. writes start out fast, can be horizontally scaled
4. much less opportunity for logic in the db
8

22. Hot Potato API - Why MongoDB?
Mongo comparison
1. reads start out fast, can be externally cached, can also be
horizontally scaled
2. mongo doesn’t support joins - data starts de-normalized
3. writes start out fast, can be horizontally scaled
4. much less opportunity for logic in the db
5. map/reduce for pre-materialization/aggregation, which can be
horizontally scaled
8

23. Hot Potato API - Why MongoDB?
Mongo comparison
1. reads start out fast, can be externally cached, can also be
horizontally scaled
2. mongo doesn’t support joins - data starts de-normalized
3. writes start out fast, can be horizontally scaled
4. much less opportunity for logic in the db
5. map/reduce for pre-materialization/aggregation, which can be
horizontally scaled
6. indexes can be removed over time as usage becomes more KV-like
8

24. Hot Potato API - Why MongoDB?
Mongo comparison
1. reads start out fast, can be externally cached, can also be
horizontally scaled
2. mongo doesn’t support joins - data starts de-normalized
3. writes start out fast, can be horizontally scaled
4. much less opportunity for logic in the db
5. map/reduce for pre-materialization/aggregation, which can be
horizontally scaled
6. indexes can be removed over time as usage becomes more KV-like
The gradual scaling process is more natural with Mongo. Horizontal
scaling is not an after-thought or bolt on addition. Mongo works
perfectly well as a KV store if you ever get to that point.
8

25. Hot Potato API - Why Scala?
• Runs on JVM (Stable, Fast)
• Access to Java’s many libraries
• Language benefits
• Terse
• Supports Immutability
• Functional
• Concurrent
• Easy to write DSLs
9

26. Three key Scala features
• Pattern Matching / Case classes
• Implicit conversions
• Actors
10

27. Building a DSL for Mongo
Documents are a flexible building block:
• Insertion
• Updates
• Queries
• Sorting
• parts of Map / Reduce
• Indexes
11

28. Building a DSL for Mongo
Goals
• Stay close to the MongoDB Java API
• Keep it flexible
• Focus on document creation
• New documents, queries, updates
• Protect against mis-named fields
Key classes and objects
• Collection - wraps MongoDB DBCollection
• MongoAST - defines the types for building Mongo documents
• MongoDSL - defines DSL syntax
12

29. Usage Patterns
Asynchronous atomic updates
• Simple observer pattern implemented with Lift Actors
• HpActor, HpActorPool, Notifier
• Main action object with a Notifier
• Example: checkins
• Listeners that use MongoDB atomic updates
• Example: EventAdjuster
13