Riak 2.0 : For Beginners, and Everyone Else

1. RIAK 2.0
Engin Yöyen
October, 2014

2. What is Riak
• Key/Value Store
• hashTable[‘D018319’] = ‘Influenza’
• diseases = hashTable[‘D018319’]

3. Buckets
• There is no global namespace.
• All keys must belong to a bucket.
• diseases[‘D018319’] = ‘Influenza’
• symptoms[‘D018319’] = ‘cervical pains’

4. Why Bucket Types
• diseases[‘infectious’][‘D018319’] = ‘Influenza’
• diseases[‘blood-borne’][‘F023831’] = ‘Hepatitis B’
• Groups of buckets with a similar set of properties.
• diseases.props = {"search_index":"anyplace"}

5. The API

6. The API
via HTTP methods PUT, GET, POST, DELETE
Example
[hostAddress]/types/<type>/buckets/<bucket>/keys/<key>

7. PUT
curl -XPUT "http://localhost:8098/types/diseases/buckets/
infectious/keys/D018319"
-H "Content-Type:text/plain"
-d "Influenza, commonly known as, the flu, ...."
riak-admin bucket-type create diseases '{"props":
{"datatype":"set"}}'
riak-admin bucket-type activate diseases

8. PUT (2)
curl -v -XPUT "http://localhost:8098/types/diseases/buckets/
infectious/keys/D018319"
-H "Content-Type: application/json"
-d '{"desc": "Influenza, commonly known as, the flu, …."}'
Hint : http://localhost:8098/types/diseases/buckets/infectious/keys/D018319?returnbody=true

9. GET
http://localhost:8098/types/diseases/buckets/infectious/keys/
D018319

10. Post
http://localhost:8098/types/diseases/buckets/infectious/keys

11. Delete
I am sure you all figured out by now….

12. Hints

13. Hints - Buckets
http://localhost:8098/types/diseases/buckets?buckets=true
{
"buckets": [
"blood-borne",
"infectious"
]
}

14. Hints - Keys
http://localhost:8098/types/diseases/buckets/infectious/keys?keys=true
{
"keys": [
"Uhz73mNfxkEkcmFsLcgu3Kan5Dw",
"D018319",
"WapIL5eeRbnXkocmcK3YK5EDjFV",
"9CkpJtAFziWuYSTfadW2rsf4bpF"
]
}
or you can stream it
http://localhost:8098/types/diseases/buckets/infectious/keys?keys=stream

15. Hints - Properties
http://localhost:8098/types/diseases/buckets/infectious/props
{
"props": {
"name": "infectious",
"w": "quorum",
"rw": "quorum",
"r": "quorum",
"pw": 0,
"precommit": [],
"pr": 0,
"postcommit": [],
"notfound_ok": true,
"n_val": 3,
"last_write_wins": false,
"dw": "quorum",
"dvv_enabled": true,
"chash_keyfun": {
"basic_quorum": false,
"allow_mult": true,
"datatype": "set",
"active": true,
"claimant": “riak@127.0.0.1”,
…
}
}

16. Backends
• Bitcask, eLevelDB, Memory or Multi

17. The Cluster

18. The Cluster
A cluster in Riak is a managed collection of nodes
that share a common Ring.

19. The Ring
Figure : Riak Ring, (Redmond & Daily, 2014, P. 11)

20. Replication + Partitions
Figure : Replication Partitions, (Redmond & Daily, 2014, P. 10)
But how?

21. vnodes
Vnodes, are responsible
for claiming a partition in
the Riak Ring, and they
coordinate requests for
these partitions.
n_val = 3
Figure : Riak Ring, (Redmond & Daily, 2014, P. 11)
ring_creation_size = 64
favorite = 75017a36ec07fd4c377a0d2a011400ab193e61db

22. nodes & vnodes
Figure : Riak Ring, (Redmond & Daily, 2014, P. 12)

23. The CAP

24. The CAP
• Consistency (all nodes see the same data at the same
time)
• Availability (a guarantee that every request receives a
response about whether it was successful or failed)
• Partition tolerance (the system continues to operate
despite arbitrary message loss or failure of part of the
system)
CAP loosely states that you can have a C (consistent), A
(available), or P (partition-tolerant) system, but you can only
choose 2, meaning CP or AP.

25. Replication & Write & Read
• n_val=3
• w=all (consistent, high latency)
• r=all (consistent read, high latency)
• w=3,r=1(likely consistent, low latency)
per bucket basis
Figure : NRW, (Redmond & Daily, 2014, P. 14)

26. The Quorum
(floor(N/2) + 1)
• nodes A,B,C,D,E,F,G,L,K,J (Don’t count it, it is 10)
“n_val” : "5"
“w" : "quorum"
• w = floor(5/2)+1
• w=3
• PUT may respond successfully after writing to (A,B,C) and data will be
eventually replicated to (D,E)
why?

27. What about Delete?

28. What about Delete?
• Object are marked as deleted with a
marker(tombstone) and another process(reaper) will
finish deleting the marked objects
• A delete is actually a read and a write
• Checking for the existence of a key(e.g.
MapReduce,List) is not enough to know if an object
exists.
• Check for tombstone metadata(X-Riak-Deleted=true)

29. A Delete Operation
• Client makes the request
• Vnode : update vclock, store with tombstone
• delete_mode = keep, 3000 ms, immediate
• backend flags it as a tombstone (Bitcask or
LevelDB) or wipes it immediately (in-memory)
all r/w/n_val parameters must be met

30. Data Types

31. Data Types
Allows RIAK to make sense of data(a data-aware system),
so it can solve the conflicts.
• Bucket Level: counters, sets, maps
• In Maps : flags(enable, disable) and registers
With and Without DataTypes
[hostAddress]/types/<type>/buckets/<bucket>/keys/<key>
[hostAddress]/types/<type>/buckets/<bucket>/datatypes/<key>

32. Data Type : Set
Adding symptoms to the Influenza
http://localhost:8098/types/diseases/buckets/infectious/datatypes/
D018319
{"add_all":["Headaches", "Fatigue"]}
{"add_all":["Cough", "Sore throat"]}
{"remove": "Cough"}
Result : {["Headaches","Fatigue","Sore throat"]}

33. Data Type : Map
Adding more than symptoms to the Influenza
http://localhost:8098/types/diseases/buckets/infectious/datatypes/D018319
{
"update": {
"symptoms_set": {
"add_all": [
"Headaches",
"Fatigue",
"Sore throat"
]
}
}
}
{
"update": {
"name_register": "Influenza",
"icd-9_register": "487"
}
}

34. Data Type : Map
Adding and removing items from set
{
"update": {
"symptoms_set": {
"remove": "Headaches",
"add": "Fever"
}
}
}
Maps within maps
{
"update": {
"virus_classification_map": {
"update": {
"influenzavirus_A_register": "Some Text",
"influenzavirus_B_register": "Some Text",
"influenzavirus_C_register": "Some Text"
}
}
}
}

35. Data Type : Map
And result is
{
"type": "map",
"value": {
"virus_classification_map": {
"influenzavirus_C_register": "Some Text",
"influenzavirus_B_register": "Some Text",
"influenzavirus_A_register": "Some Text"
},
"symptoms_set": [
"Fatigue",
"Fever",
"Sore throat"
],
"name_register": "Influenza",
"icd-9_register": "487"
},
"context": "g2wAAAABaAJtAAAACCMJ/vlUQtGHYQRq"
}

36. Last but not least
• Search
• Security
• Hooks
• A Little Riak Book : http://littleriakbook.com/

37. Questions!

38. Reference
• Redmond,
E.,
&
Daily,
J.
(2014).
A
Little
Riak
Book.
• Riak
Docs.
http://docs.basho.com/riak/latest/