Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
SPEED UP YOUR SYMFONY2 
APPLICATION AND BUILD 
AWESOME FEATURES WITH REDIS
HELLO WORLD 
• Ricard Clau, born and grown up in Barcelona 
• Server engineer at Another Place Productions 
• Symfony2 lov...
AGENDA 
• REDIS introduction, data types and commands 
• Getting started with Redis in PHP / Symfony2 
• Options out of th...
INTRODUCTION TO REDIS 
And some demystifications
WHAT IS REDIS? 
• REmote DIctionary Server 
• Created in 2009 by Salvatore Sanfilipo (@antirez) 
• Open source (https://gi...
TO ME REDIS IS LIKE...
ONLY IN-MEMORY? 
• Your data needs to fit in your memory 
• It has configurable persistence: RDB snapshots, AOF 
persisten...
INTERESTING FEATURES 
• Master-slave replication 
• Pipelining to improve performance 
• Transactions (kind of with MULTI ...
COMMANDS 
http://redis.io/commands
NO QUERIES 
•We communicate with Redis via commands 
• No indexes, no schemas, just structures under a key 
• Very good do...
DATA TYPES 
• Strings (binary-safe) up to 512 Mb 
• Lists of elements sorted by insertion order (max 2^32 - 1) 
• Sets sup...
STRINGS COMMANDS 
• GET, SET, STRLEN, APPEND 
• GETRANGE, SETRANGE 
• Bitmap operations: GETBIT, SETBIT, BITCOUNT 
• Count...
LISTS COMMANDS 
Key S1 S2 S3 … SN 
• Order of insertion matters 
• Easy to implement Queues and Stacks 
• RPUSH, LPUSH, RP...
SETS COMMANDS 
Key 
S1 
S5 
S3 
S2 
S4 
• Collection of unique unordered elements 
• SCARD (Size), SADD, SREM, SISMEMBER, ...
HASHES COMMANDS 
Key 
K1 V1 
K2 V2 
K3 V3 
K4 V4 
• Many key-value pairs under the same key 
• HSET, HGET, HGETALL, HLEN, ...
SORTED SETS COMMANDS 
S4 - 1000 
Key S2 - 20 
S3 - 15 
S1 - 0 
• Collection of unique elements ordered by score 
• ZCARD (...
HYPERLOGLOG COMMANDS 
• Counting unique things in massive data sets using a very 
small amount of memory 
• PFADD, PFCOUNT...
GETTING STARTED 
Easy steps
PHP CLIENTS 
• Predis (PHP) vs phpredis (PHP extension) 
• Predis is very mature, actively maintained, feature complete, 
...
EASY STEPS (I) 
$ composer require snc/redis-bundle 1.1.* 
$ composer require predis/predis 0.8.* 
or install the phpredis...
EASY STEPS (II) 
Add the bundle to the Symfony2 kernel
EASY STEPS (III) 
Add some magic to your config.yml
EASY STEPS (IV) 
You can now use Redis as a Symfony2 service! 
! 
In your controller extending 
SymfonyBundleFrameworkBund...
EASY STEPS (V) 
Or inject it to your services via your services.xml
REDIS-CLI AND MONITOR 
• Redis-cli to connect to a Redis instance 
• Experiment with http://redis.io/commands 
•With Monit...
SNCREDISBUNDLE 
So many options out of the box!
CLIENTS EVERYWHERE
PHP SESSION SUPPORT 
! 
TTL is the session lifetime, Redis expires it for you 
Session Locking implementation (not optimal...
LOCKING PROBLEM 
• Problem in heavy-write applications: 2 almost concurrent 
requests trying to update same records 
• Con...
DOCTRINE CACHING 
Implemented with SET / SETEX / GET 
Be careful with expirations! 
No expiring 
Possibly better 
use APC ...
MONOLOG (I) 
Implemented with LISTS using RPUSH 
Be careful with memory!
MONOLOG (II) 
And add a new monolog handler using the service 
Level recommended at least warning
SWIFTMAILER SPOOL 
Implemented with LISTS using RPUSH 
Serialized Swift_Mime_Messages in the list
COOKBOOKS 
From Real World applications
REDIS AS A QUEUE 
• Using Lists and LPUSH / RPOP instructions 
•We can have different languages accessing data 
• I used t...
REAL-TIME DASHBOARDS 
•We can use hashes to group many stats under one key 
• Most of the times we have counters: 
HINCRBY...
LEADERBOARDS 
• Super-easy with Redis, heavy consuming with other systems 
• Each board with a single key, using sorted se...
WHO IS ONLINE? (I) 
•We can use SETS to achieve it! 
• One set for every minute, SADD <minute> <userId> 
Time +1m +2m +3m ...
WHO IS ONLINE? (II) 
• Online users == everyone active in the last N minutes 
• SUNION <now> <now-1> <now-2> ... 
• SUNION...
FRIENDS ONLINE? 
• If we store user´s friends in a SET with key friends-<user-id> 
• Friends online: SINTERSTORE “friends-...
SHARDING / CLUSTERING 
This is when it gets tricky…
NEEDS TO FIT IN MEMORY 
• Redis Cluster will be available in 3.0 (currently RC1) 
• Proxy servers: Twemproxy (also for mem...
CONFIGS WARNING! 
! 
The default behaviour is RandomDistributionStrategy! 
PHPRedis has a RedisArray class for client-side...
1-99999 100000-199999 
200000-299999 300000-399999 
id % 4 == 0 id % 4 == 1 
id % 4 == 2 id % 4 == 3 
Easy to predict scal...
hash(key) % 4 == 0 hash(key) % 4 == 1 
hash(key) % 4 == 2 hash(key) % 4 == 3 
Distribution also depending on hash algorith...
Many Redis instances in one 
PRESHARDING 
Maybe overengineered 
initial server 
! 
When needed, just split it 
into more s...
REDIS CLUSTER 
• Shards the dataset among N nodes 
• Has a responsive failover in order to survive certain failures 
(this...
FINAL THOUGHTS 
When and when not to consider Redis
REDIS IS PERFECT FOR... 
• Intensive read-write data applications 
• Temporary stored data 
• Data that fits in memory 
• ...
BUT IS NOT SUITABLE WHEN.. 
• Big data sets, archive data 
• Relational data (RDBMS are absolutely fine to scale) 
•We don...
SPECIAL THANKS 
• Ronny López (@ronnylt) & Alonso Vidales (@alonsovidales) 
• Salvatore Sanfilippo (@antirez) 
• Henrik We...
QUESTIONS? 
• Twitter: @ricardclau 
• E-mail: ricard.clau@gmail.com 
• Github: https://github.com/ricardclau 
• Blog about...
Nächste SlideShare
Wird geladen in …5
×

Speed up your Symfony2 application and build awesome features with Redis

24.234 Aufrufe

Veröffentlicht am

Redis is an extremely fast data structure server that can be easily added to your existing stack and act like a Swiss army knife to help solve many problems that would be extremely difficult to workaround with the traditional RDBMS. In this session we will focus on what Redis is, how it works, what awesome features we can build with it and how we can use it with PHP and integrate it with Symfony2 applications making them blazing fast.

Veröffentlicht in: Software

Speed up your Symfony2 application and build awesome features with Redis

  1. 1. SPEED UP YOUR SYMFONY2 APPLICATION AND BUILD AWESOME FEATURES WITH REDIS
  2. 2. HELLO WORLD • Ricard Clau, born and grown up in Barcelona • Server engineer at Another Place Productions • Symfony2 lover and PHP believer (sometimes…) • Open-source contributor, sometimes I give talks • Twitter @ricardclau / Gmail ricard.clau@gmail.com
  3. 3. AGENDA • REDIS introduction, data types and commands • Getting started with Redis in PHP / Symfony2 • Options out of the box in SncRedisBundle • Cookbooks implemented in Real World Applications • Sharding data and the new Redis Cluster
  4. 4. INTRODUCTION TO REDIS And some demystifications
  5. 5. WHAT IS REDIS? • REmote DIctionary Server • Created in 2009 by Salvatore Sanfilipo (@antirez) • Open source (https://github.com/antirez/redis) • Advanced in-memory key-value data-structure server • Part of the NoSQL movement, think back to structures!
  6. 6. TO ME REDIS IS LIKE...
  7. 7. ONLY IN-MEMORY? • Your data needs to fit in your memory • It has configurable persistence: RDB snapshots, AOF persistence logs, combine both or no persistence at all • Think like memcached on steroids with persistence • http://redis.io/topics/persistence • “Memory is the new disk, disk is the new tape”
  8. 8. INTERESTING FEATURES • Master-slave replication • Pipelining to improve performance • Transactions (kind of with MULTI ... EXEC) • Scripting with LUA (~stored procedures) • Iterators for Keyspaces and SETS and HASHES (>= 2.8) !
  9. 9. COMMANDS http://redis.io/commands
  10. 10. NO QUERIES •We communicate with Redis via commands • No indexes, no schemas, just structures under a key • Very good documentation and sandbox • All commands have their complexity documented
  11. 11. DATA TYPES • Strings (binary-safe) up to 512 Mb • Lists of elements sorted by insertion order (max 2^32 - 1) • Sets supporting unions, intersections and diffs (max 2^32 - 1) • Hashes key-value pairs (max 2^32 - 1) • Sorted sets automatically ordered by score (max 2^32 - 1) • HyperLogLog to compute cardinality of BIG sets (2^64)
  12. 12. STRINGS COMMANDS • GET, SET, STRLEN, APPEND • GETRANGE, SETRANGE • Bitmap operations: GETBIT, SETBIT, BITCOUNT • Counter operations: INCR, DECR, INCRBY, DECRBY • If the value is a number, Redis tries to store it efficiently
  13. 13. LISTS COMMANDS Key S1 S2 S3 … SN • Order of insertion matters • Easy to implement Queues and Stacks • RPUSH, LPUSH, RPOP, LPOP • Blocking versions BLPOP, BRPOP, BRPOPLPUSH with timeout
  14. 14. SETS COMMANDS Key S1 S5 S3 S2 S4 • Collection of unique unordered elements • SCARD (Size), SADD, SREM, SISMEMBER, SMEMBERS • Intersections, Unions and Diffs: SINTER, SUNION, SDIFF, SINTERSTORE, SUNIONSTORE, SDIFFSTORE • Iterators with SSCAN (Redis >= 2.8)
  15. 15. HASHES COMMANDS Key K1 V1 K2 V2 K3 V3 K4 V4 • Many key-value pairs under the same key • HSET, HGET, HGETALL, HLEN, HEXISTS • Counters HINCR, HINCRBY, HINCRBYFLOAT • Iterators with HSCAN (Redis >= 2.8)
  16. 16. SORTED SETS COMMANDS S4 - 1000 Key S2 - 20 S3 - 15 S1 - 0 • Collection of unique elements ordered by score • ZCARD (Size), ZADD, ZREM, ZSCORE • Ranking: ZRANGE, ZREVRANGE (optional WITHSCORES) • Iterators with ZSCAN (Redis >= 2.8)
  17. 17. HYPERLOGLOG COMMANDS • Counting unique things in massive data sets using a very small amount of memory • PFADD, PFCOUNT, PFMERGE • HyperLogLog has an standard error of 0.81% • PF prefix in honor of Philippe Flajolet
  18. 18. GETTING STARTED Easy steps
  19. 19. PHP CLIENTS • Predis (PHP) vs phpredis (PHP extension) • Predis is very mature, actively maintained, feature complete, extendable, composer friendly and supports all Redis versions • Phpredis is faster, but not always backwards compatible • Network latency is the biggest performance killer • http://redis.io/clients
  20. 20. EASY STEPS (I) $ composer require snc/redis-bundle 1.1.* $ composer require predis/predis 0.8.* or install the phpredis extension https://github.com/nicolasff/phpredis
  21. 21. EASY STEPS (II) Add the bundle to the Symfony2 kernel
  22. 22. EASY STEPS (III) Add some magic to your config.yml
  23. 23. EASY STEPS (IV) You can now use Redis as a Symfony2 service! ! In your controller extending SymfonyBundleFrameworkBundleControllerController.php
  24. 24. EASY STEPS (V) Or inject it to your services via your services.xml
  25. 25. REDIS-CLI AND MONITOR • Redis-cli to connect to a Redis instance • Experiment with http://redis.io/commands •With Monitor we can watch live activity!
  26. 26. SNCREDISBUNDLE So many options out of the box!
  27. 27. CLIENTS EVERYWHERE
  28. 28. PHP SESSION SUPPORT ! TTL is the session lifetime, Redis expires it for you Session Locking implementation (not optimal) Added not so many months ago
  29. 29. LOCKING PROBLEM • Problem in heavy-write applications: 2 almost concurrent requests trying to update same records • Concurrent AJAX requests for the same user timeline R1read R1 R2 R2save R2read R1save R1 updates are lost!!!!!
  30. 30. DOCTRINE CACHING Implemented with SET / SETEX / GET Be careful with expirations! No expiring Possibly better use APC entities data (annotations / xml / yaml) RecordSets Setting TTL explicitly DQL parsing
  31. 31. MONOLOG (I) Implemented with LISTS using RPUSH Be careful with memory!
  32. 32. MONOLOG (II) And add a new monolog handler using the service Level recommended at least warning
  33. 33. SWIFTMAILER SPOOL Implemented with LISTS using RPUSH Serialized Swift_Mime_Messages in the list
  34. 34. COOKBOOKS From Real World applications
  35. 35. REDIS AS A QUEUE • Using Lists and LPUSH / RPOP instructions •We can have different languages accessing data • I used this to send OpenGraph requests to Facebook (REALLY slow API) with Python daemons • If you need some advanced message queuing features check RabbitMQ (@cakper last month´s talk) and others
  36. 36. REAL-TIME DASHBOARDS •We can use hashes to group many stats under one key • Most of the times we have counters: HINCRBY “stats" <key> <increment> • HMGET “stats” <key1> <key2> ... to retrieve values and HMSET “stats” “stat1” <value1> “stat2” <value2> to set them • HGETALL to get the full hash
  37. 37. LEADERBOARDS • Super-easy with Redis, heavy consuming with other systems • Each board with a single key, using sorted sets • ZINCRBY “rankXXX” <increment> <userId> • Top N ZREVRANGE “rankXXX” 0 N [WITHSCORES] • You can store several millions of members under 1 key
  38. 38. WHO IS ONLINE? (I) •We can use SETS to achieve it! • One set for every minute, SADD <minute> <userId> Time +1m +2m +3m 2 2 7 4 1 2 3 1 1 5 3 1 SUNION 1 2 3 5 7 4
  39. 39. WHO IS ONLINE? (II) • Online users == everyone active in the last N minutes • SUNION <now> <now-1> <now-2> ... • SUNIONSTORE “online” <now> <now-1> <now-2> ... • Number of users: SCARD “online” • Obtain online users with SMEMBERS “online”
  40. 40. FRIENDS ONLINE? • If we store user´s friends in a SET with key friends-<user-id> • Friends online: SINTERSTORE “friends-<userid>-online” “online” “friends-<userid>” ! ! 1 7 3 5 2 4 15 1 ! • Imagine how you would do it without Redis! 9 3 10 7 ONLINE FRIENDS-12 SINTER FRIENDS-12-ONLINE
  41. 41. SHARDING / CLUSTERING This is when it gets tricky…
  42. 42. NEEDS TO FIT IN MEMORY • Redis Cluster will be available in 3.0 (currently RC1) • Proxy servers: Twemproxy (also for memcached) • Client-side partitioning (supported in many clients) • Sharding strategies (range vs hash partitioning) • Presharding can help scaling horizontally
  43. 43. CONFIGS WARNING! ! The default behaviour is RandomDistributionStrategy! PHPRedis has a RedisArray class for client-side sharding but this is not supported yet in the bundle!
  44. 44. 1-99999 100000-199999 200000-299999 300000-399999 id % 4 == 0 id % 4 == 1 id % 4 == 2 id % 4 == 3 Easy to predict scale Load not properly distributed Load distributed ok Fairly easy to reshard RANGE PARTITIONING May work in some cases, not a silver bullet
  45. 45. hash(key) % 4 == 0 hash(key) % 4 == 1 hash(key) % 4 == 2 hash(key) % 4 == 3 Distribution also depending on hash algorithm Complicated resharding HASH PARTITIONING Better distribution... but still issues If few keys, it also becomes useless HASH CRC16 CRC32 MD5 SHA1 MURMUR3 ! DIST MODULUS KETAMA
  46. 46. Many Redis instances in one PRESHARDING Maybe overengineered initial server ! When needed, just split it into more servers and you don´t need resharding!!! ! Be careful configuring memory limits
  47. 47. REDIS CLUSTER • Shards the dataset among N nodes • Has a responsive failover in order to survive certain failures (this was partially covered already with Sentinel) • Ability to reshard keys internally • Neither CP nor AP, eventually consistent, not very resistant to network partitions in some scenarios • Many disagree with some of the decisions and tradeoffs
  48. 48. FINAL THOUGHTS When and when not to consider Redis
  49. 49. REDIS IS PERFECT FOR... • Intensive read-write data applications • Temporary stored data • Data that fits in memory • Problems that fit Redis built-in data types • Predictability in performance needed
  50. 50. BUT IS NOT SUITABLE WHEN.. • Big data sets, archive data • Relational data (RDBMS are absolutely fine to scale) •We don´t know how we will access data • Reporting applications (no where clauses) • It gets tricky for distributed applications (replication, clustering) • ALWAYS choose the right tool for the job!
  51. 51. SPECIAL THANKS • Ronny López (@ronnylt) & Alonso Vidales (@alonsovidales) • Salvatore Sanfilippo (@antirez) • Henrik Westphal (https://github.com/snc) • Danielle Alessandri (https://github.com/nrk) • Nicolas Favre-Felix (https://github.com/nicolasff) • Of course, to all of you for being here today!
  52. 52. QUESTIONS? • Twitter: @ricardclau • E-mail: ricard.clau@gmail.com • Github: https://github.com/ricardclau • Blog about PHP and Symfony2: http://www.ricardclau.com

×