2. CONTENT
⢠SNS games and SQL-based databases
⢠NoSQL technology and Couchbase
⢠NoSQL does not come without challenges
⢠SNS Storage Engine (SSE)
4. SNS games characteristics
⢠Huge amount of concurrent requests but
require low response time
⢠Accounts can be stored separately
â No need for centralized storage
â In most cases, no need to put strict constrains on
data relationship
5. Native limitations of SQL-based DBMS
⢠Centralized fundamentally
â Vertical scale up issue
⢠Schema
â High risk (and cost) for updates
⢠Normalized data
â Unnecessary overhead: join tables, locking, data
constrain check,âŚ
7. Native limitations of SQL-based DBMS
⢠SQL processing overhead at both DBMS and
client side.
⢠Most data accesses end up at hard-disk
â Very challenging to meet low response time
â Internal caching does not help much
⢠Hard to distributed data across multiple-
servers
9. NoSQL technology
⢠Persistent distributed hash-table
⢠Active set resides on RAM
â Extremely fast response time
⢠Horizontal scale up
⢠Raw and direct data access
â set, get, add, inc, dec : no overhead
10. NoSQL technology
Key Value
Jack.Gold 50123
Jack.Exp 4670
Jack.Coin 700
Peter.Gold 7050
Peter.Exp 20005
Peter.Coin 1
Key Value Key Value Key Value
Peter.Gold 7050 Jack.Gold 50123 Peter.Coin 1
Jack.Exp 4670 Jack.Coin 700
Peter.Exp 20005
11. Active set on RAM
CLIENT
ACTIVE SET ON RAM
Lazy write
HDD
12. Couchbase server
⢠Based on membase technology
⢠Distributed
⢠Replica
⢠Since 1.8, have native client for PHP
⢠Bucket types
â Couchbase (persistent)
â Memcache (memory only)
15. Architecture and design issues
⢠Transition from relational database design to
key-value design
â Account data => keys : how ?
⢠Only minimum support for
locking, concurrency control
â add : failed if exists - mutex
â cas : read get cas, write failed if cas is out-dated
16. Architecture and design issues
⢠No transaction support
â Data corruption becomes so easy!
⢠No high-level data support (e.g. list,queue,âŚ)
⢠No tools for raw data viewing / editing
17. Pitfalls
⢠Too much freedom for developers
â Anyone can add / modify any key any time
⢠Epic key design mindset
â One key for all : bad performance, concurrency
control is a true night mare
⢠Abuse the power of set
â Never fail ! Developer LOVE it !
20. What is SSE ?
⢠A thin âlayerâ between developers and the
all-mighty Couchbase
â SSE is simply a PHP library
⢠Provide better support for locking and
concurrency control
â Basic support for : Begin â update - commit
⢠Provide high-level data structures
â Collection, queue, stack, integer (gold), inc-only
integer (exp), binary flags (quest)âŚ
21. What is SSE ?
⢠Minimize the risk of weak concurrency support
â Ability to rollback pending writes
⢠Schema
â Limit freedom of developers!
â No more nightmare for backup and raw data
view/editing
⢠Buffers to eliminate repeated read / writes
25. Multi-instance architecture
⢠Replica is too costly to performance
⢠One node failed means cluster failed
⢠Adding nodes requires rebalance
â Only good when having clusters with large
number of nodes (more than 20 nodes)
26. Multi-instance architecture
⢠One instance for index (user-to-instance
mapping)
â Use APC on logic servers to cache / reduce load
to index instance
⢠Many instances of data
â Dynamically adjust weight on each instance base
on average load of instance
â Node failure only affects part of the user-base
27. Multi-instance architecture
Game Logic Game Logic Game Logic Game Logic
APC APC APC APC
Index Data Data Data
Instance Instance 1 Instance 2 Instance 3
29. How good is SSE for us ?
⢠No more data loss due to concurrency
⢠No more data corruption
⢠No mysterious bugs due to un-intended
writes
⢠Reduce more than 3 times workload of server
developers