Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Redis & Groovy integration for caching and messaging
1. Redis & Groovy & Grails
by Ted Naleid
http://naleid.com
Monday, June 20, 2011
2. “Redis is a collection of data
structures exposed over the
network”
from: http://nosql.mypopescu.com/post/5403851771/what-is-redis
Monday, June 20, 2011
3. key/value store
like memcached on steroids
Monday, June 20, 2011
4. Strings, Integers,
Lists, Hashes,
Sets & Sorted Sets
(& commonly expected operations with each data type)
Monday, June 20, 2011
8. “Memory is the new Disk,
Disk is the new Tape”
- Jim Gray
Monday, June 20, 2011
9. Relative Latency
CPU Register - 1x
L2 Cache - 10x
Memory - 100x
Disk - 10,000,000x
analogy from “Redis - Memory as the New Disk” - Tim Lossen &
http://en.wikipedia.org/wiki/Orders_of_magnitude_(speed)
Monday, June 20, 2011
10. CPU Register
1 yard
photo: http://www.flickr.com/photos/limonada/904754668/
Monday, June 20, 2011
15. % telnet localhost 6379
Escape character is '^]'.
set foo bar
+OK
get foo
$3
bar
rpush mylist first
:1
rpush mylist second
:2
lrange mylist 0 -1
*2
$5
first
$6
second
Monday, June 20, 2011
16. clients for every* language
*well not every language, but all the popular/semi-popular ones, you can easily write one if
your language doesn’t have one
Monday, June 20, 2011
26. other uses...
distributed locks, tag clouds, session tokens,
auto-complete prefixes, API rate limiting, leaderboards,
capped logs, random set items, A/B testing data storage,
unique per user product pricing/sorting
Monday, June 20, 2011
46. Hashes
hkeys (hash keys)
Redis REPL Groovy
bar baz
foo
qux quxx
> hkeys foo redis.hkeys("foo")
1) "bar" <= [bar, qux]
2) "qux"
bar qux
Monday, June 20, 2011
47. Sets
sadd (set add)
Redis REPL Groovy
> sadd m1 jan redis.sadd("m1", "jan")
m1 jan
(integer) 1 <= 1
Monday, June 20, 2011
48. Sets
sadd (set add)
Redis REPL Groovy
feb
> sadd m1 feb redis.sadd("m1", "feb")
m1
(integer) 1 <= 1
jan
Monday, June 20, 2011
49. Sets
sismember (membership test)
Redis REPL Groovy
feb
m1
jan
> sismember m1 jan redis.sismember("m1", "jan")
(integer) 1 <= true
1
Monday, June 20, 2011
50. Sets
sismember (membership test)
Redis REPL Groovy
feb
m1
jan
> sismember m1 mar redis.sismember("m1", "mar")
(integer) 0 <= false
0
Monday, June 20, 2011
51. Sets
smembers (get full set)
Redis REPL Groovy
feb
m1
jan
> smembers m1 redis.smembers("m1")
1) "feb" <= [feb, jan]
2) "jan"
feb
jan
Monday, June 20, 2011
52. Sets
sinter (set intersection)
Redis REPL Groovy
feb feb
m1 m2
jan mar
> sinter m1 m2 redis.sinter("m1", "m2")
1) "feb" <= ["feb"]
feb
Monday, June 20, 2011
53. Sets
sdiff (set difference)
Redis REPL Groovy
feb feb
m1 m2
jan mar
> sdiff m1 m2 redis.sdiff("m1", "m2")
1) "jan" <= ["jan"]
jan
Monday, June 20, 2011
54. Sets
sunion (set union)
Redis REPL Groovy
feb feb
m1 m2
> sunion m1 m2 jan mar
1) "mar" redis.sunion("m1", "m2")
2) "jan" mar <= ["mar", "jan", "feb"]
3) "feb"
jan
feb
Monday, June 20, 2011
55. Sorted Sets
zadd (add with score)
Redis REPL Groovy
> zadd z1 1 jan redis.zadd("z1", 1, "jan")
z1 1 jan
(integer) 1 <= 1
Monday, June 20, 2011
56. Sorted Sets
zscore (score for member)
Redis REPL Groovy
1 jan
z1 2 feb
> zscore z1 feb 3 mar redis.zscore("z1", "feb")
"2" <= 2.0
2
Monday, June 20, 2011
57. Sorted Sets
zrange (sorted subset)
Redis REPL Groovy
1 jan
z1 2 feb
> zrange z1 0 1 withscores
1) "jan" 3 mar redis.zrangeWithScores("z1", 0, 1)
2) "1" <= [["jan", 1], ["feb", 2]]
3) "feb"
4) "2"
1 jan
2 feb
Monday, June 20, 2011
58. Sorted Sets
zrangebyscore (subset having score range)
Redis REPL Groovy
1 jan
z1 2 feb
> zrangebyscore z1 2 3 withscores
1) "feb" 3 mar redis.zrangeByScoreWithScores("z1",2,3)
2) "2" <= [["feb", 2], ["mar", 3]]
3) "mar"
4) "3"
2 feb
3 mar
Monday, June 20, 2011
62. Producer
pushes work on a list with lpush
@Grab('redis.clients:jedis:2.0.0')
redis = new redis.clients.jedis.Jedis("localhost")
args.each { redis.lpush("welcome-wagon", it) }
Monday, June 20, 2011
63. Consumer
uses blpop (blocking left pop from list)
@Grab('redis.clients:jedis:2.0.0')
redis = new redis.clients.jedis.Jedis("localhost")
println "Joining the welcome-wagon!"
while (true) {
def name = redis.blpop(0, "welcome-wagon")[1]
println "Welcome ${name}!"
}
Monday, June 20, 2011
64. Mass Producer
srandmember to randomly pick female name from set
@Grab('redis.clients:jedis:2.0.0')
redis = new redis.clients.jedis.Jedis("localhost")
if (!redis.exists("female-names")) {
new File("./female-names.txt").eachLine {redis.sadd("female-names",it)}
}
for (i in 1..100000) {
redis.lpush("welcome-wagon", redis.srandmember("female-names"))
if (i % 1000 == 0) println "Adding $i"
}
female-names.txt from: http://antirez.com/post/autocomplete-with-redis.html
Monday, June 20, 2011
69. RedisTagLib
<redis:memoize key="mykey" expire="3600">
<!--
insert expensive to generate GSP content here
content will be executed once, subsequent calls
will pull from redis (redis.get(“mykey”)) till the key expires
-->
</redis:memoize>
Monday, June 20, 2011
70. RedisService
Spring bean wraps pool connection
// overrides propertyMissing and methodMissing to delegate to redis
def redisService
redisService.foo = "bar"
assert "bar" == redisService.foo
redisService.sadd("months", "february")
assert true == redisService.sismember("months", "february")
Monday, June 20, 2011
71. RedisService
template methods manage pooled Redis connection
redisService.withRedis { Jedis redis ->
redis.set("foo", "bar")
}
Monday, June 20, 2011
73. RedisService
String memoization
redisService.memoize("my-key") { Jedis redis ->
// expensive operation we only want to execute once
}
def ONE_HOUR = 3600 // with optional timeout in seconds
redisService.memoize("my-key-with-timeout", ONE_HOUR) { Jedis redis ->
// expensive operation we want to execute every hour
}
Monday, June 20, 2011
74. RedisService
Domain Class memoization (stores IDs hydrates from DB)
def key = "user:$id:friends-books"
redisService.memoizeDomainList(Book, key, ONE_HOUR) { redis ->
// expensive process to calculate all friend’s books
// stores list of Book ids, hydrates them from DB
}
Monday, June 20, 2011
75. Example
Showing Products with Sort/Filter/Pagination Criteria
Monday, June 20, 2011
76. Other Memoization Methods
memoizeHash, memoizeHashField,
memoizeScore (sorted set score)
Monday, June 20, 2011
79. Can be used in conjunction
with Hibernate
Monday, June 20, 2011
80. Partial support for GORM
including Dynamic Finders, Criteria, Named Queries and “Transactions”
Monday, June 20, 2011
81. Limitations
It requires explicit index mapping on fields you want to query
package com.example
class Author {
String name
static mapWith = "redis"
static hasMany = [books: Book]
static mapping = {
name index:true
}
}
Monday, June 20, 2011
82. Under The Covers
MONITOR output for new Author(name: "Stephen King").save()
1308027697.922839 "INCR" "com.example.Author.next_id"
1308027697.940021 "HMSET" "com.example.Author:1" "name" "Stephen King" "version" "0"
1308027697.940412 "SADD" "com.example.Author.all" "1"
1308027697.943318 "SADD" "com.example.Author:id:1" "1"
1308027697.943763 "ZADD" "com.example.Author:id:sorted" "1.0" "1"
1308027697.944911 "SADD" "com.example.Author:name:Stephen+King" "1"
Monday, June 20, 2011