2011 03-31 Riak Stockholm Meetup

Overview
Mårten Gustafson
Stockholm Riak Meetup #1
2011-03-31

What is Riak
and what’s the agenda?

Decentralized key-value store
A database ideally suited for web applications
A ﬂexible map/reduce engine

http://www.ﬂickr.com/photos/linneberg/4481309196/

Operations
HTTP Client
PUT /bucket/key
GET /bucket/key
DELETE /bucket/key

Item Item
Item
Item Item
Item Item Item

Bucket Bucket Bucket

Riak Key/Value Store

An entry

• lives in a bucket
• has a key
• has a value

An entry
arbitrary name

bucket
• has a key
• has a value

An entry
arbitrary name

bucket
• has a key arbitrary name
key
• has a value

An entry
arbitrary name
forms the path
bucket to the value
key
• has a value

An entry
arbitrary name
forms the path
bucket to the value
key
• has a value
value
a binary blob
and mime type

An entry
arbitrary name
forms the path
bucket to the value
key
• has a value
value
a binary blob
and mime type
= Store anything, yay!

Store anything
bucket meetup

Store anything
bucket meetup

key foo.html mime text/html
<html><body>
<h1>
<a href=”bar.html”>foo</a>
</h1>
</body></html>

Store anything
bucket meetup

key foo.html mime text/html key bar.html
<html><body> mime text/html
<h1>
<a href=”bar.html”>foo</a> <html><body>
</h1> <h1>bar!</h1>
</body></html> </body></html>

Store anything
bucket meetup

key foo.html mime text/html key bar.html
<html><body> mime text/html
<h1>
<a href=”bar.html”>foo</a> <html><body>
</h1> <h1>bar!</h1>
</body></html> </body></html>

http://127.0.0.1:8088/riak/meetup/foo.html

Storage

Bitcask InnoDB DETS File system Balanced trees LRU ETS

Storage


disk based, ram based,
durable not durable

Storage

default

commo disk based, ram based,
durable not durable

The Ring
12 1
11 2

10 3

9 4

8 5
7 6

ring size = 12

Consistent Hashing
12 1
11 2

10 3

9 4

8 5
7 6

Read

“I want “
requires us to know:
where is on the ring?

Cluster

One Ring size to rule them all, One Ring size to
ﬁnd them, One Ring size to bring them all and in
the cluster bind them...

Cluster

node A

node B

node C

ring size = 12
instances = 3
ring size / nodes = ~slices per instances

Cluster - Read
node A node B node C

Cluster - Read
I can haz ?


Hm, hashes to a
slice of the ring
owned by node C.

Cluster - Read
I can haz ?


Okidoki, now
Hey C! I need where’s he...a yeah
in my fourth slice

Cluster - Read
I can haz ?


...network partitions?

...network partitions?
node A node B X node C

...failed nodes?

...failed nodes?
node A node B

...concurrent writes?

client 1 client 2

...concurrent writes?
node A
?
node B node C

hey, A!
save for me
client 1 client 2

hey, C!
save for me

Livin’ on the web

• concurrent users
• unreliable networks
• node failures

The N to the R to the W to the DW and the RW

N_VAL
Client

Bucket

Node A Node B Node C

N_VAL 1
Client

Bucket


N_VAL 2
Client

Bucket


N_VAL 3
Client

Bucket


R
Client GET /bucket/key?R=1

Bucket


R

Bucket


R

Agree?

Bucket



Number of copies
ie. distribute to N nodes

Read
ie. have R nodes agree


Number of copies
ie. distribute to N nodes

Read
ie. have R nodes agree

Write
Number of copies ie. ack’d by
ie. distribute to N nodes W nodes

Durable write
Read ie. persistently written by
ie. have R nodes agree DW nodes

Write

Durable write
Read ie. persistently written by
ie. have R nodes agree DW nodes

Write Read-write
ie. persistently deleted by
RW nodes

The Quorum
([node count] / 2) + 1

Bucket properties

http://127.0.0.1:8088/riak/meetup/

I don’t care!

allow_mult = false
last_write_wins = true

I do care!

allow_mult = true

I do care!

• Resolve conﬂicts in application logic
• Conﬂicts exposed as siblings beneath a key
• Response is HTTP 300 Multiple Choice
• Served as mime/multipart

Example
HTTP/1.1 300 Multiple Choices

X-Riak-Vclock: a85hYGBgzWDKBVIsrLnh3BlMiYx5rAymfeeO8EGFWRLl30G==

Content-Type: multipart/mixed; boundary=ZZ3eyjUllBi7GXRRMJsUublFxjn

Content-Length: 368

--ZZ3eyjUllBi7GXRRMJsUublFxjn

Content-Type: text/plain

Tuesday

--ZZ3eyjUllBi7GXRRMJsUublFxjn

Content-Type: text/plain

Thursday

--ZZ3eyjUllBi7GXRRMJsUublFxjn--

Map / Reduce

• Javascript or Erlang
• Exposed in the HTTP API

Map / Reduce count words

function(v) {
var words = v.values[0].data.toLowerCase().match('w*','g');
var counts = [];
for(var word in words)
if (words[word] != '') {
var count = {};
count[words[word]] = 1;
counts.push(count);
}
return counts;
}

Map / Reduce count words

function(values) {
var result = {};
for (var value in values) {
for(var word in values[value]) {
if (word in result)
result[word] += values[value][word];
else
result[word] = values[value][word];
}
}
return [result];
}

Map & Reduce count words

Put this in your POST request and let Riak smoke it

{"inputs":"bucket", "query":[{"map":{"language":"javascript",
"source":"function(v) { var words = v.values[0].data.toLowerCase().match(/
w*/g); var counts = []; for(var word in words) if (words[word] != '')
{ var count = {}; count[words[word]] = 1; counts.push(count); } return
counts; }"}},{"reduce":{"language":"javascript", "source":"function(values)
{ var result = {}; for (var value in values) { for(var word in
values[value]) { if (word in result) result[word] += values[value][word];
else result[word] = values[value][word]; } } return [result]; }"}}]}

function(v) {
var words = v.values[0].data.toLowerCase().match('w*','g');
var counts = [];
for(var word in words)
if (words[word] != '') {
var count = {};
count[words[word]] = 1;
counts.push(count);
}
return counts;
}


function(values) {
var result = {};
for (var value in values) {
for(var word in values[value]) {
if (word in result)
result[word] += values[value][word];
else
result[word] = values[value][word];
}
}
return [result];
}


Links

• Non-enforced
• Traversable

The whole enchilada
HTTP API Riak Search Luwak

Riak Core
Membership management Work distribution
leave/join

Cluster state Partitioning
gossip protocol (consistent hashing, hinted handoff)

Erlang / OTP

Try it

downloads.basho.com

brew install riak

Web admin @ github.com/gmaurice/Riaktive

Resources

Riak Fast Track @ wiki.basho.com

#riak @ freenode

github.com/basho/

Thanks for listening

Mårten Gustafson
@martengustafson

http://marten.gustafson.pp.se/

marten.gustafson@gmail.com

2011 03-31 Riak Stockholm Meetup

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie 2011 03-31 Riak Stockholm Meetup

Ähnlich wie 2011 03-31 Riak Stockholm Meetup (20)

Mehr von Mårten Gustafson

Mehr von Mårten Gustafson (6)

Kürzlich hochgeladen

Kürzlich hochgeladen (19)

2011 03-31 Riak Stockholm Meetup