1. How to choose the right
NoSQL Technology
Josh Butts - 7/6/2011

2. Agenda
• Spoilers
• Evaluation criteria
• Winners & Losers
• Gotchas
• Questions & Discussion

3. Spoilers

4. Spoilers
• We picked MongoDB and Redis

5. Spoilers
• We picked MongoDB and Redis
• We love Redis to death

6. Spoilers
• We picked MongoDB and Redis
• We love Redis to death
• We’re in a platonic relationship with
MongoDB

7. Spoilers
• We picked MongoDB and Redis
• We love Redis to death
• We’re in a platonic relationship with
MongoDB
• Maybe one day it can be something more

8. Where to start?

9. Where to start?
• First, why are you even considering
this?

10. Where to start?
• First, why are you even considering
this?
• Very, very clearly define the problem

11. Where to start?
• First, why are you even considering
this?
• Very, very clearly define the problem
• You already know SQL. Are you sure
it isn’t the right choice?

12. Building Criteria

13. Building Criteria
• Make a list of real criteria

14. Building Criteria
• Make a list of real criteria
• These should be points which will help you
differentiate

15. Building Criteria
• Make a list of real criteria
• These should be points which will help you
differentiate
• Hint: “Fast, flexible, good language
support” are not useful

16. Building Criteria
• Make a list of real criteria
• These should be points which will help you
differentiate
• Hint: “Fast, flexible, good language
support” are not useful
• These are the common denominator

17. Use case #1 at
Offers.com

18. Use case #1 at
Offers.com
• Log impressions of content on web
pages

19. Use case #1 at
Offers.com
• Log impressions of content on web
pages
• Light caching

20. Use case #1 at
Offers.com
• Log impressions of content on web
pages
• Light caching
• Data de-normalization

21. Our requirements

22. Our requirements
• Reads & writes need to be blazing fast

23. Our requirements
• Reads & writes need to be blazing fast
• Data persistence is reasonably
important

24. Our requirements
• Reads & writes need to be blazing fast
• Data persistence is reasonably
important
• Need to implement incrementally

25. The contenders

26. The contenders
• In-memory key-value stores

27. The contenders
• In-memory key-value stores
• Memcached

28. The contenders
• In-memory key-value stores
• Memcached
• Membase

29. The contenders
• In-memory key-value stores
• Memcached
• Membase
• Redis

30. The contenders
• In-memory key-value stores
• Memcached
• Membase
• Redis
• Tokyo Tyrant

31. Why Redis

32. Why Redis
• Redis is damn cool

33. Why Redis
• Redis is damn cool
• In memory + disk storage

34. Why Redis
• Redis is damn cool
• In memory + disk storage
• As fast as Memcached

35. Why Redis
• Redis is damn cool
• In memory + disk storage
• As fast as Memcached
• “Remote data structures”

36. Why Redis
• Redis is damn cool
• In memory + disk storage
• As fast as Memcached
• “Remote data structures”
• This is where the magic happens

37. Extras with Redis

38. Extras with Redis
• Most libraries provide a session
handler

39. Extras with Redis
• Most libraries provide a session
handler
• Super-easy to install

40. Extras with Redis
• Most libraries provide a session
handler
• Super-easy to install
• Ability to treat it like memcached

41. Extras with Redis
• Most libraries provide a session
handler
• Super-easy to install
• Ability to treat it like memcached
• memory model not quite the same

42. Extras with Redis
• Most libraries provide a session
handler
• Super-easy to install
• Ability to treat it like memcached
• memory model not quite the same
• In fact, not the same at all

43. Why not the others?

44. Why not the others?
• Memcached has no disk persistance

45. Why not the others?
• Memcached has no disk persistance
• I can’t rely on the data (easily)

46. Why not the others?
• Memcached has no disk persistance
• I can’t rely on the data (easily)
• Membase seems pointless-ish
(compared to redis) for small-to-
medium scale

47. Use Case #2 at
Offers.com

48. Use Case #2 at
Offers.com
• Process (diff / update) 10 million
SKU-level items from 100 partners

49. Use Case #2 at
Offers.com
• Process (diff / update) 10 million
SKU-level items from 100 partners
• Do this once per day

50. Use Case #2 at
Offers.com
• Process (diff / update) 10 million
SKU-level items from 100 partners
• Do this once per day
• Deal with disparate data schemas

51. Use Case #2 at
Offers.com
• Process (diff / update) 10 million
SKU-level items from 100 partners
• Do this once per day
• Deal with disparate data schemas
• Deal with missing / null data nicely

52. The Contenders

53. The Contenders
• Document-oriented Databases

54. The Contenders
• Document-oriented Databases
• MongoDB

55. The Contenders
• Document-oriented Databases
• MongoDB
• CouchDB

56. The Contenders
• Document-oriented Databases
• MongoDB
• CouchDB
• Cassandra

57. Why Documents?

58. Why Documents?
• Data arrives in various XML formats

59. Why Documents?
• Data arrives in various XML formats
• No one likes XML except Java devs

60. Why Documents?
• Data arrives in various XML formats
• No one likes XML except Java devs
• Need flexible schema

61. Why Documents?
• Data arrives in various XML formats
• No one likes XML except Java devs
• Need flexible schema
• Need searching / querying by various
fields

62. Why MongoDB

63. Why MongoDB
• Fast

64. Why MongoDB
• Fast
• Outstanding connectivity

65. Why MongoDB
• Fast
• Outstanding connectivity
• Schema free

66. Why MongoDB
• Fast
• Outstanding connectivity
• Schema free
• Good community support

67. Why MongoDB
• Fast
• Outstanding connectivity
• Schema free
• Good community support
• Ad-hoc queries

68. Why MongoDB
• Fast
• Outstanding connectivity
• Schema free
• Good community support
• Ad-hoc queries
• Extremely low learning curve

69. Why MongoDB
• Fast
• Outstanding connectivity
• Schema free
• Good community support
• Ad-hoc queries
• Extremely low learning curve
• They sent us stickers

70. Why not the others

71. Why not the others
• We’re pretty sure Cassandra is just a
Facebook troll project

72. Why not the others
• We’re pretty sure Cassandra is just a
Facebook troll project
• CouchDB is brilliant but queries can
be tedious without a plan

73. Downsides of Mongo

74. Downsides of Mongo
• I still don’t trust that my data is 100%
safe

75. Downsides of Mongo
• I still don’t trust that my data is 100%
safe
• Aggregates and map/reduce are
cumbersome compared to CouchDB

76. Downsides of Mongo
• I still don’t trust that my data is 100%
safe
• Aggregates and map/reduce are
cumbersome compared to CouchDB
• Lack of good GUI tools for exploring
data

77. Where we failed

78. Where we failed
• We didn’t clearly articulate the
problem

79. Where we failed
• We didn’t clearly articulate the
problem
• We built a brilliant product catalog
parser

80. Where we failed
• We didn’t clearly articulate the
problem
• We built a brilliant product catalog
parser
• We forgot it had to get to MySQL
eventually

81. Why is this harder than
a traditional RDBMS?

82. Why is this harder than
a traditional RDBMS?
• Vendor lock in (model)

83. Why is this harder than
a traditional RDBMS?
• Vendor lock in (model)
• Vendor lock in (feature set)

84. Why is this harder than
a traditional RDBMS?
• Vendor lock in (model)
• Vendor lock in (feature set)
• Vendor lock in (fanboyism)

85. How does SQL solve
this?

86. How does SQL solve
this?
• ANSI SQL

87. How does SQL solve
this?
• ANSI SQL
• Adapter patterns

88. How does SQL solve
this?
• ANSI SQL
• Adapter patterns
• Abstraction layers

89. How does SQL solve
this?
• ANSI SQL
• Adapter patterns
• Abstraction layers
• ORMs

90. With NoSQL, you must
choose!

91. With NoSQL, you must
choose!
• There’s no useful way to abstract
NoSQL systems to the point of code
portability

92. With NoSQL, you must
choose!
• There’s no useful way to abstract
NoSQL systems to the point of code
portability
• ODMs might ease the pain

93. Questions?