新浪微博开放平台中的 Redis 实践
•
16 gefällt mir•1,261 views
2011年10月，Qcon 杭州，开放平台话题，新浪微博开放中的 Redis 实践，唐福林
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie 新浪微博开放平台中的 Redis 实践
Ähnlich wie 新浪微博开放平台中的 Redis 实践 (20)
Mehr von fulin tang
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
新浪微博开放平台中的 Redis 实践
- 1. Redis @ http://weibo.com/tangfl http://fulin.org
- 2. • Redis • Redis • • •
- 3. Redis • in memory (database?) • data can dump to disk • many useful data structure • FAST both read and write • we start use from 2.0, now 2.4
- 4. =feed+ +
- 5. =feed+ + • feed • •
- 6. =feed+ +
- 7. =feed+ + • • •
- 8. =feed+ +
- 9. =feed+ + • • • @ • @ •
- 10. =feed+ +
- 11. =feed+ + • • • @ • @ •
- 12. =feed+ +
- 13. =feed+ + • • • @ • @ •
- 14. =feed+ +
- 15. =feed+ + • • • @ • @ •
- 16. =feed+ +
- 17. =feed+ + •feed • mysql • mc
- 18. =feed+ + •
- 19. =feed+ + • mysql: relation.following • fromuid, touid, addtime • addtime desc select * from following where fromuid=? order by • addtime desc select * from following where touid=? order by • fromuid, touid
- 20. =feed+ + • mysql: relation.following relation.follower • fromuid, touid, addtime • addtime desc select * from following where fromuid=? order by • desc select * from follower where touid=? order by addtime •
- 21. =feed+ + • • • • •
- 22. =feed+ + • ta • ta • ta •
- 23. =feed+ + • • c/java • • •
- 24. =feed+ + • redis • hash : • key : user id • fields : friends ids • value : add time
- 25. =feed+ + • redis • hash : • hset fromuid.following touid addtime • hset touid.follower fromuid addtime • hgetAll fromuid. following • hgetAll touid.follower ? • 11,704,598 @Wed Sep 7 21:46:33 CST 2011
- 26. =feed+ + • hash-max-zip-size • 64 -> 256 1/3 • cpu • hgetAll cost too much cpu • add mc • high delay
- 27. =feed+ +
- 28. =feed+ + • redis • cache ? waste too much mem • storage ? • rdb may lost data • aof r/w too slow, recover too slow • all data in mem, waste money • HA : master slave ? NO WAY • memory fragment
- 29. =feed+ +
- 30. =feed+ + • • mysql binlog >> queue >> Java Processor >> redis • mysql binlog >> trigger >> redis
- 31. =feed+ + • • redis@weibo for now: • TB • growing fast
- 32. =feed+ + • •@ mysql modified + innodb • following list of a user in one column • still under dev
- 33. =feed+ + •
- 34. =feed+ + • • • •@ • •
- 35. =feed+ +
- 36. =feed+ + • • • @ • • • js
- 37. =feed+ + • mc + queue + mysql • mysql • mc mysql • mc
- 38. =feed+ + • • • • •
- 39. =feed+ + • redis • k-v , 100 byte per k-v • mc • • hash , hget pipeline slow
- 40. =feed+ + • redis • rdb ? may lost data • aof ? grow too fast (4G/day) • bgsave/bgrewriteaof influence parent
- 41. =feed+ +
- 42. =feed+ + • redis rolling • • • • key
- 43. =feed+ + • • rediscounter @ • array , not linked list • malloc all mem when start • hash key to position • write disk: asyn & slow down • add position to aof file
- 44. =feed+ + • • rediscounter + innodb • auto roll cold data to disk
- 45. =feed+ + • • @TimYang • @XiaoJunHong • • redis (delay) - mc (throughput) - java hash map
- 46. • • cache • storage
- 47. • • • • + •
- 48. • • • / • •
- 49. • • • rdb or aof
- 50. • • • rehash
- 51. • HA / Cluster • Redis • Jedis
- 52. • CPU • Redis • hset with big hash-max-zip-size • hgetAll • mc
- 53. • • • • •
- 54. Thanks @ http://weibo.com/tangfl http://fulin.org
- 55. Q &A PS. We are hiring ! contact me via @