Slides from my lightning talk at NoSQL Live Boston.

1. Versatile Storage Options
with Tokyo Cabinet
NoSQL Live
March 11, 2010
Flinn Mueller

2. Tokyo Cabinet
• Mikio Hirabayashi
• Mixi.jp
• LGPL
• C99/Posix
• Successor to QDBM
• mmap

3. Hash DB
• Key/Value
• Strings
• Extensions (Lua)
• Map/Reduce
• Forward Matching Keys

4. B+ Tree Store
• Key/Value
• Duplicate Keys
• Forward Matching Keys
• (Perfect Tag Store)

5. Fixed Length DB
• Key/Value
• Keys As Natural Number
• Value Length Limited
• Faster than Hash or B+ Tree
• Range Query

6. Table Store
• Tuple/Document Store
• Query (condition, order, limit)
• Full Text Search
• Union, Intersect, Diff
• Parallel Query

7. How I Use Tokyo
• Like Memcache++
• Like a Queue
• Like a Counter
• Like a Tag Cloud

8. {}

9. Like Memcache++
require 'tokyo_tyrant'
t = TokyoTyrant::DB.new
t['example.com/lorem'] = "Lorem..."
t['example.com/lorem']
# => "Lorem..."
t['example.com/lorem/ipsum'] = "Ipsum..."
t.fwmkeys('example.com/')
# => ['example.com/lorem/ipsum', 'example.com/lorem']

10. Q

11. Like a Queue
require 'tokyo_tyrant'
tdb = TokyoTyrant::Table.new
tdb[tdb.genuid] = {
'url' => "http://example.com",
'queue' => "1m",
'created_at' => Time.now.to_i,
'origin' => Socket.hostname
}

12. Like a Queue
require 'tokyo_tyrant'
tdb = TokyoTyrant::Table.new
tasks = tdb.find{ |q|
q.condition :queue, :streq, '1m'
q.order_by :created_at
}.each{ |task| run_process(task) }

13. i++

14. Like a Counter
require 'tokyo_tyrant'
COUNTER_DB = TokyoTyrant::DB.new
...
COUNTER_DB.add_int(id)
# => 82169

15. zomg
pictures
lolcatz
funny

16. Like a Tag Cloud
require 'tokyo_tyrant'
TAG_DB = TokyoTyrant::BDB.new
...
tags = ['foo', 'bar']
tags.collect!{ |tag| "tag:#{tag}:articles" }
hash_tags = TAG_DB.getlist(tags)
article_ids = hash_tags.values.collect.flatten
article_ids = article_ids.uniq.sort.reverse[0..4]
articles = TAG_DB.mget(article_ids)

17. Related Content
articles = TAG_DB.mget(article_ids)
unpack = proc{ |k,v| v = MessagePack.unpack(v); v['__id'] = k; v }
articles = articles.collect(&unpack).compact
articles = articles.sort_by{ |v| v['__id'] }
articles.reverse!
content = Mustache.render(@@template, :articles => articles)
response = [200, {"Content-Type" => "text/html"}, [content]]

18. Versatile Storage Options
• Like Memcache++
• Like a Queue
• Like a Counter
• Like a Tag Cloud

19. More
http://1978th.net/tokyocabinet
http://1978th.net/tokyotyrant/
http://groups.google.com/group/tokyocabinet-users/
http://tokyocabinetwiki.pbworks.com/
http://github.com/actsasflinn/ruby-tokyotyrant
http://github.com/jmettraux/rufus-tokyo/