Riak ( http://wiki.basho.com ), a Dynamo-inspired, open-source key/value datastore, was built to scale from a single machine to a 100+ server cluster without driving you or your operations team crazy. This presentation discusses the characteristics of Riak that become important in small, medium, and large clusters.

1. Riak from Small to Large
Strange Loop · October 2010
Rusty Klophaus (@rklophaus)
Basho Technologies

2. Riak is a Dynamo-inspired,
open-sourced, key/value datastore
built to scale predictably and easily.
2

3. Riak is a Dynamo-inspired,
open-sourced, key/value datastore
built to scale predictably and easily.
Your ops guy: calm, relaxed, and wearing a party hat.
3

4. NoSQL
4

5. 5
http://www.flickr.com/photos/ghewgill/129950124

6. NoSQL has changed like Keanu hasn’t.
6
http://www.youtube.com/watch?v=nEubt6HpGhs

7. Companies, Funding, Customers, Awareness...
7
http://www.flickr.com/photos/free-stock/4791385567

8. “NoSQL? Yeah, it’s this really cool technology,
you probably haven’t heard of it yet.”
8
http://www.flickr.com/photos/slava/285199203

9. “Johnson! Get me a memo on
this ‘No sequal’ thing!”
9
http://www.flickr.com/photos/proimos/4045973322

10. Why?
10

11. Because NoSQL is driven by
narcissism, voyeurism, and
materialism.

12. The catalyst for NoSQL
is the data
generated or consumed
by humans.
12

13. Photos, music, and video...
13

14. Constantly connected devices...
14

15. Social Networks...
15
http://www.flickr.com/photos/hanspoldoja/5001818922

16. Your Grandmama...
16
http://www.flickr.com/photos/mega/7358278

17. Requirements That Confound an RDBMS
Large Data Objects Changing Schemas
BLOBs? Auto-generated DDL?
Generic Columns?
High-Availability Systems Non-Standard Access
Bi-Directional Replication? Recursive SQL?
Log shipping?
High-Capacity Systems
Shards?
Vertical Partitioning?
17

18. The roots of NoSQL are practical,
not academic.
18

19. Turning Point
Scratch an Itch ➮ Build a Company
19

20. Four Important Things
1. Broad Knowledge
Learn from the past, both DB *and* non-DB worlds.
• VP Technical Services, Akamai
• Senior Architect, Akamai
• Chief of Operations, Snapfish
• Release Manager, Engine Yard
• Bob Ippolito (CTO/CoFounder MochiMedia)
• Dr. Eric Brewer (CAP Theorum)
20

21. Four Important Things
2. Strong Community
Stay in touch with real problems.
Mark Phillips
Basho Community Manager
21

22. Four Important Things
3. Agility
Evolve to solve new problems.
22

23. Four Important Things
4. Restraint
No solution can do it all well.
23

24. Back to Riak
24

25. What characteristics of Riak become
important at different cluster sizes?
25

26. Single Box Riak
Simplicity
26

27. Single Box Riak
Simple Key/Value Store, Flexible Schema
client = RiakClient(host, port)
bucket = client.bucket('mybucket')
obj = bucket.new('foo', 'bar')
obj.store()
obj2 = bucket.get(‘foo’)
27

28. Single Box Riak
Clients in Many Languages
28

29. Single Box Riak
Predictable Development Interface
Riak Riak
Riak === Riak Riak
Riak
Development Production
29

30. Single Box Riak
Configurable Buckets
!=
Profile Audio
Bucket Bucket
30

31. Single Box Riak
Links (Lightweight Data Relations)
Profile Projects Comments
31

32. Small Riak Cluster
Parallelism and Power
32

33. Small Riak Cluster
Expanding the Cluster
bin/riak-admin join riak@hostname
33

34. Small Riak Cluster
Expanding the Cluster
bin/riak-admin join riak@hostname
34

35. Small Riak Cluster
Expanding the Cluster
bin/riak-admin join riak@hostname
Hello ladies, can your
datastore grow like mine?
*monocle smile*
34

36. Small Riak Cluster
Distributed Queries
35

37. Small Riak Cluster
Distributed Queries
36

38. Small Riak Cluster
Javascript-Based Map/Reduce
Map
Map
Reduce
Map
Keys Reduce Output
Map
Reduce
...
Map
37

39. Small Riak Cluster
Well-Behaved HTTP
Client
Client
nginx
Browser
Client Riak Riak
Squid Riak Riak
Riak
38
nginx proxy config - http://gist.github.com/323048

40. Small Riak Cluster
Riak Full-Text Search
bin/search-cmd install mybucket
"converse AND category:shoes"
Riak Riak
Riak Riak
Riak
39
nginx proxy config - http://gist.github.com/323048

41. Large Riak Cluster
Operations
40

42. Large Riak Cluster
No Special Nodes
Special
Nodes
41

43. Large Riak Cluster
Laugh at Machine Failure
Riak Riak
Riak
Riak
X
Riak
42

44. Large Riak Cluster
Scale by Adding Nodes
Riak Riak
Riak
Riak Riak Riak
Riak
Riak
Riak Riak
Riak
Riak
43

45. Large Riak Cluster
Packages or Self-Contained Installation
sudo dpkg -i riak_0.13.0-2_amd64.deb
-or-
unzip riak.zip
riak/bin/riak start
44

46. $0.00

47. Order now, and $0.00
we’ll include an
extra Riak for free!*
*Offer limited to open-
source editions of Riak.

48. Enterprise ($$$ / ~10’s of boxes)
• On-Call Support 24x7x365
• Management Tools
• SNMP Monitoring
• Multi-site Replication
47

49. Enterprise ($$$ / ~10’s of boxes)
• On-Call Support 24x7x365
• Management Tools
• SNMP Monitoring
• Multi-site Replication
48

50. Thanks! Questions?
Next Steps
Read more at http://wiki.basho.com
Download binaries from http://downloads.basho.com
Get source code from http://hg.basho.com
Join the public mailing list at riak-users@lists.basho.com
Mailing list archives at http://riak.markmail.org
Thanks!
Rusty Klophaus (@rklophaus)
Mark Phillips (@pharkmillups)
49

51. Additional Slides

52. Tutorial
Get Riak
Connect with a Client
Store Data
Store an Object with Links
Linkwalking
Map/Reduce
51

53. Get Riak
Download
http://downloads.basho.com/riak
Start Riak
cd riak
bin/riak start
52

54. Connect with Python
# Code is at http://hg.basho.com
import riak
# Connect
client = riak.RiakClient('127.0.0.1', 8098)
53

55. Store Data
mybucket = client.bucket('mybucket')
# Create an object...
obj = mybucket.new('myobject')
obj.set_data({ 'foo' : 1, 'bar' : 2 })
obj.store()
# Read the object...
obj = mybucket.get('myobject')
print obj.get_data()
# Or, open a web browser...
http://127.0.0.1:8098/riak/mybucket/myobject
54

56. Store an Object with Links
bands = client.bucket('bands')
albums = client.bucket('albums')
members = client.bucket('members')
# Store a band, link to album and members...
obj = bands.new('Winger')
.add_link(albums.new('Pull', 1275922).store())
.add_link(albums.new('IV', 542731).store())
.add_link(albums.new('Karma', 200170).store())
.add_link(members.new('Kip Winger').store())
.add_link(members.new('Reb Beach').store())
.add_link(members.new('John Roth').store())
.add_link(members.new('Rod M.').store())
.store()
55

57. Linkwalking
# Get the albums...
albums = obj.link('albums').run()
# Get the songs (assumes data is present)...
songs = obj.link('albums').link('songs').run()
# Get the members...
members = riak.MapReduce(client)
.add('bands', 'Winger')
.link('members')
.run()
56

58. Map/Reduce
# Count the number of sales...
result = obj
.link('albums')
.map("function(v) { return [v.values[0].data]; }")
.reduce("Riak.reduceSum")
.run()
57