Link Walking with Riak

1. Link Walking with Riak (KV) Susan Potter / Finsignia June 25, 2011

2. Riak Ecosystem Riak Search Riak KV Riak Core

7. % whoami # curl http://susanpotter.net/whoami/windycitydb2011.json • 15 years RDBMS • 3-4 years KV databases (CouchDB and now Riak) • 1 year Redis • Using graph database in next part of platform

11. Scope of Talk • Problem Domain • not Consistent Hashing or DHTs • Linking Objects • not Vector Clocks • Traversing Links • not Gossip Protocol • Link Maintainence • not Hinted Handoff • not Read Repair • not Eventual Consistency

21. Problem Domain

22. Problem Domain

23. Linking[1/2] curl -X PUT -H ’Content-type: text/json’ -H ’Link: </riak/artist/Lady-Gaga>; riaktag=“performer”’ -H ’Link: </riak/artist/DJ-Snake>; riaktag=“producer”’ -H ’Link: </riak/artist/DJ-White-Shadow>; riaktag=“producer”’ ... -H ’Link: </riak/artist/Fernando-Garibay>; riaktag=“producer”’ http://host:8098/riak/album/Born-This-Way data ...

28. Linking[2/2] • Two Dimensional both direction and type/tag • 0..* links per object of any type/tag • Metadata traversing not load, parse, then lookup cycle • Model sufﬁciency otherwise graph database might be more useful • Link to other buckets (in same Riak cluster)

33. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 • GET /riak/album/Born-This-Way/artist,performer,1 • GET /riak/album/Born-This-Way/artist,producer,1 • GET /riak/album/Born-This-Way/_,performer,1 • GET /riak/album/Born-This-Way/_,producer,1

34. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 • GET /riak/album/Born-This-Way/artist,producer,1 • GET /riak/album/Born-This-Way/_,performer,1 • GET /riak/album/Born-This-Way/_,producer,1

35. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 • GET /riak/album/Born-This-Way/artist,producer,1 • GET /riak/album/Born-This-Way/_,performer,1 • GET /riak/album/Born-This-Way/_,producer,1

36. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 => artists linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/artist,producer,1 • GET /riak/album/Born-This-Way/_,performer,1 • GET /riak/album/Born-This-Way/_,producer,1

37. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 => artists linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/artist,producer,1 • GET /riak/album/Born-This-Way/_,performer,1 • GET /riak/album/Born-This-Way/_,producer,1

38. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 => artists linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/artist,producer,1 => artists linked to album with tag/type “producer” • GET /riak/album/Born-This-Way/_,performer,1 • GET /riak/album/Born-This-Way/_,producer,1

39. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 => artists linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/artist,producer,1 => artists linked to album with tag/type “producer” • GET /riak/album/Born-This-Way/_,performer,1 • GET /riak/album/Born-This-Way/_,producer,1

40. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 => artists linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/artist,producer,1 => artists linked to album with tag/type “producer” • GET /riak/album/Born-This-Way/_,performer,1 => object linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/_,producer,1

41. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 => artists linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/artist,producer,1 => artists linked to album with tag/type “producer” • GET /riak/album/Born-This-Way/_,performer,1 => object linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/_,producer,1

42. Traversing Links[1/4] • GET /riak/album/Born-This-Way/artist,_,1 => all artists linked to album regardless of link tag/type • GET /riak/album/Born-This-Way/artist,performer,1 => artists linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/artist,producer,1 => artists linked to album with tag/type “producer” • GET /riak/album/Born-This-Way/_,performer,1 => object linked to album with tag/type “performer” • GET /riak/album/Born-This-Way/_,producer,1 => object linked to album with tag/type “producer”

43. Traversing Links[2/4] • GET /riak/album/A/artist,_,1/artist,collaborator,1 • GET /riak/album/A/artist,performer,0/artist,collaborator,1 • GET /riak/album/A/artist,producer,0/album,release,1

44. Traversing Links[2/4] • GET /riak/album/A/artist,_,1/artist,collaborator,1 • GET /riak/album/A/artist,performer,0/artist,collaborator,1 • GET /riak/album/A/artist,producer,0/album,release,1

45. Traversing Links[2/4] • GET /riak/album/A/artist,_,1/artist,collaborator,1 => all artists associated with album A and all artists that have collaborated with these artists • GET /riak/album/A/artist,performer,0/artist,collaborator,1 • GET /riak/album/A/artist,producer,0/album,release,1

48. Traversing Links[2/4] • GET /riak/album/A/artist,_,1/artist,collaborator,1 => all artists associated with album A and all artists that have collaborated with these artists • GET /riak/album/A/artist,performer,0/artist,collaborator,1 => all “collaborators” of “performers” on album A excluding “performers” • GET /riak/album/A/artist,producer,0/album,release,1

51. Traversing Links[2/4] • GET /riak/album/A/artist,_,1/artist,collaborator,1 => all artists associated with album A and all artists that have collaborated with these artists • GET /riak/album/A/artist,performer,0/artist,collaborator,1 => all “collaborators” of “performers” on album A excluding “performers” • GET /riak/album/A/artist,producer,0/album,release,1 => all albums that “producers” on album A have released

52. Traversing Links[3/4] artist,performer,1 • Bucket name e.g. artist, album • Tag (or type) name e.g. performer, producer • Return (or “keep”) useful in nested traversals • Wildcarding ’_’ for bucket and tag means “any” • Nesting traverse on multiple levels • Multiple Sources? how to walk links from multiple objects?

56. Traversing Links[3/4] _,_,0 • Bucket name e.g. artist, album • Tag (or type) name e.g. performer, producer • Return (or “keep”) useful in nested traversals • Wildcarding ’_’ for bucket and tag means “any” • Nesting traverse on multiple levels • Multiple Sources? how to walk links from multiple objects?

57. Traversing Links[3/4] _,_,0/album,release,1 • Bucket name e.g. artist, album • Tag (or type) name e.g. performer, producer • Return (or “keep”) useful in nested traversals • Wildcarding ’_’ for bucket and tag means “any” • Nesting traverse on multiple levels • Multiple Sources? how to walk links from multiple objects?

58. Traversing Links[3/4] • Bucket name e.g. artist, album • Tag (or type) name e.g. performer, producer • Return (or “keep”) useful in nested traversals • Wildcarding ’_’ for bucket and tag means “any” • Nesting traverse on multiple levels • Multiple Sources? how to walk links from multiple objects?

59. curl -X POST -H “Content-type: application/json” http://host:8098/mapred data @- {"inputs":[ ["artist","Ke$ha"], ["artist", "Lady+Gaga"] ], "query":[ {"link": { "tag": "release", "bucket": "album" }}, {"map":{ "language":"javascript", "source":"function(v) return [v]; "} } ] } ^D

62. Link Maintenance • Commit Hooks can use either pre- or post-commit hooks • Automatic Backlinks can use commit hooks to generate backlinks automatically • Trigger Messages links could only be used as triggers • Delete Cleanup post-commit hook with backlinks can clean up links to new orphans automatically

66. Possibilities Object metadata extensions?

67. Limitations Not a graph database!

68. References[1/2] • Riak Wiki http://wiki.basho.com/ • Riak Mailing List riak-users@lists.basho.com • Riak Contrib http://contrib.basho.com/ • Riak Code Repos http://github.com/basho

72. References[2/2] • Secondary index support in Riak? Introducing Secondary Indices: http://www.oscon.com/oscon2011/public/schedule/detail/19806 • riak_core applications e.g. Riak Search, riak_zab, riak_pipe • Comet KV database supports Active Storage Objects (ASO), handlers http://vanish.cs.washington.edu/pubs/osdi2010comet.pdf

75. Credits • Basho & Riak logos Courtesy of Basho • Icy chained fence photo http://www.flickr.com/photos/existentist/342177746/ • Frozen chain photo http://www.flickr.com/photos/skedonk/4197921511/ • Hook and chain photo http://www.flickr.com/photos/37491467@N07/3871760093/

76. Questions? Figure: http://www.ﬂickr.com/photos/42682395@N04/ @SusanPotter

77. Questions? Figure: http://www.ﬂickr.com/photos/42682395@N04/ @SusanPotter

Link Walking with Riak

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Link Walking with Riak

Ähnlich wie Link Walking with Riak (20)

Mehr von Susan Potter

Mehr von Susan Potter (7)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Link Walking with Riak