An overview of Redis for Philly.rb on 2010-02-17

1. Redis
REmote DIctionary Server
Ryan Findley, Philly.rb, 2010-02-16

2. Disclaimer
• I’m not using this in production (yet).

3. Kind of like Memcache
• Key-value store
• All data lives in memory
• Keys can expire (or not)
• Fast, Light-weight

4. And More
• Persistence
• Multiple databases
• Queryable keyspace
• Support for integer counters
• Higher level data structures
• Atomic operations
• Ability to paginate lists without mutating them
• Master-slave replication
• Optional VM feature (new)

5. More:
Command Reference: http://
code.google.com/p/redis/wiki/
Other Goodness
CommandReference
• Written in C (C99 standard)
• No dependencies
• Detailed command reference that lists time
complexity in Big-O notation.
• Very easy to build / install (it’ll probably “just
work”)

6. Persistence(1/2) -
Snapshotting
• This is the default behavior
• Periodic, asynchronous dump to disk
• Can be every N changes or every N seconds
• For example, every 15 min if at least 1 change
and every 5 min if at least 10 changes
• Since data is written asynchronously, data can
be lost during a crash.

7. Persistence(2/2) - Append
Only File
See more here:
http://code.google.com/p/redis/
wiki/AppendOnlyFileHowto
• Available as of version 1.1
• Works like a journal
• Every write command is logged ASAP
• Commands replayed when the server is
restarted
• Configurable speed/safety (safest by default)

8. More here:
http://code.google.com/
p/redis/wiki/
SelectCommand
http://code.google.com/
p/redis/wiki/
MoveCommand
Multiple Databases
• Use the SELECT command; 0-15 are valid by
default, but you can add more in redis.conf
• Useful for segmenting key namespaces,
especially when key queries are needed
• MOVE command can be used as a locking
primitive

9. Command Reference

10. More:
http://code.google.com/p/
redis/wiki/
IntroductionToRedisDataTypes
Data Structures
• Strings
• Strings-as-integers (used by INCR/DECR)
• List of Strings
• Set of Strings (unique list)
• Sorted Set of Strings (and weights) (ZSET)

11. Command overview
• Strings: get, set, increment, decrement
• Lists: push, pop, length, range, trim
• Sets: add, remove, move, length, intersect
union, diff, random
• Other: save, lastsave can be used to force &
verify disk persistence

12. More:
http://code.google.com/
p/redis/wiki/
RpoplpushCommand
Atomic Operations
• LPUSH / RPUSH: append to head/tail of list
• LPOP / RPOP: return & remove first/last element of list
• RPOPLPUSH: return & remove the last element of
source list and push to the head of the destination list
• GETSET: set a key to a new value and return the old
value
• MGET/MSET: get or set multiple keys to multiple values
• SMOVE: move a value from one set to another
• No way to group commands (transactions)

13. More here:
http://code.google.com/
p/redis/wiki/
ReplicationHowto
Replication
• Slaves can be used for scalability or redundancy
• A master can have multiple slaves.
• Slaves are able to accept other slave connections
• Redis replication is non-blocking on the master,
but blocking on the slave (can’t respond to
queries during initial sync)
• Slaves are able to automatically reconnect after
an outage

14. Blog post about Redis
VM:
http://antirez.com/
post/redis-virtual-
memory-story.html Optional VM feature
• Doesn’t use OS virtual memory
• Still new, may not scale well in certain situations.
• Don’t know how this will affect replication (initial
sync / re-sync may be slow, but shouldn’t block the
master)
• All keys stay in memory. Minimum req’s:
• 1 million keys ~ 160M
• 10 million keys ~ 1.6G

15. Syntax highlighted
sample here:
https://gist.github.com/
b16679cbb5b9573e078c
Setup
• git clone git://github.com/antirez/redis.git &&
cd redis
• make
• ./redis-server
• make test
• (review sample config file)

16. Redis in Ruby
• Redis library: “gem install redis”
• All Redis commands can be called like methods
on the connection object as described in the
Command Reference

17. Example Uses
• A Memcached replacement (fast)
• A work queue (sets & lists)
• Fast auto-completion (persistent, queryable)
• ORDER BY RAND() replacement (sets)

18. Memcache Replacement
• Why? Same as Memcached, but you get more “for
free”:
• Many databases from a single instance of Redis
(instead of using namespaces)
• Ability to easily backup/transfer state (dump.rdb)
• Watch live commands on a running instance with the
MONITOR command (instead of restarting with -v)
• Opportunity to use Redis for other things once
you’ve switched

19. Memcache Replacement
Note: namespace was replaced
with a DB number
• It’s pretty easy:

20. More:
Redis allows negative
http:/
Work Queue
/oxfordrepo.blogspot.com/
2010/01/usage-stats-and-redis.html
array indices to count
backwards from the end.
A VERY simple example
• LPUSH and RPOP were made for this:
1
2
3
4
5
6

21. “Not a ‘better DelayedJob’”.
More: Compare them and see what is
best for your project
http://github.com/
defunkt/resque
http://github.com/
blog/542-introducing-
Work Queue - Resque
• Created by Chris Wanstrath. Heavily inspired by
DelayedJob. Currently used by GitHub. Provides:
• A Ruby library for creating, querying, and
processing jobs
• A Rake task for starting a worker which
processes jobs
• A Sinatra app for monitoring queues, jobs, and
workers.

22. Fast Auto-Completion
• Lives in memory, doesn’t expire, persists

23. Replace MySQL ORDER
More:
BY RAND()
http://nosql.mypopescu.com/post/295433623/redis-
usecase-replacing-mysql-order-by-rand
• ORDER BY RAND() is very slow (even with a
LIMIT clause)
• Duplicating the list of IDs as a Redis set is
relatively cheap
• SRANDMEMBER is fast

24. Other Interesting Uses of
Redis
• Nginx HTTP Redis module for caching with Redis
(ngx_http_redis)
• LLOOGG.com: Realtime site usage statistics, use
alongside Google analytics
• EZMobius’ Fair Work Scheduler example
• Sikwamic: Simple Redis-backed Comet server
• RestMQ - A REST/JSON/HTTP based message queue
built on Redis
• redis-textsearch - A simple full-text search for multiple
data stores

25. Further reading
• http://code.google.com/p/redis/wiki/CommandReference
• http://antirez.com/post/redis-virtual-memory-story.html
• http://www.slideshare.net/ezmobius/redis-remote-dictionary-server
• http://www.paperplanes.de/2010/2/16/
a_collection_of_redis_use_cases.html
• http://www.dorkalev.com/2010/02/sikwamic-simple-key-value-with-
comet.html
• http://github.com/blog/542-introducing-resque
• http://nosql.mypopescu.com/post/295433623/redis-usecase-replacing-
mysql-order-by-rand
• http://github.com/nateware/redis-textsearch