5. The New Trading Card
Card sales info
Live stats and
game info
Online trading
6. Brief History of BUNT
•
2012: Introduced product on MySQL
•
•
•
Revenue model fine-tuned
Thought of new features and scaling considerations
Migrate to MongoDB for MLB 2013 opening day
•
Learn MongoDB
•
Re-design the schema
•
Write & Run migration scripts
•
Total: 4-5 months, 1-2 people
13. Big Piece 1: Processing Live
Data
• Live data is received and stored into MySQL
•
•
A heartbeat picks up the event and stores player stats into MongoDB
It then pulls from MongoDB and updates leaderboard data in Redis
14. Big Piece 2: Formatting Leaderboards for
Users
• API servers combine fan data from MongoDB with points data from Redis
•
The result is a richly-detailed set of leaderboards
15. All other
processes
• All other processes are handled by the API servers and MongoDB
Sign in, Trading, Commenting, Content Management, Purchases, Playing Cards
Storage is used for fan and player photos as well as other simple files
•
•
16. Big Piece 1
Specific reason we chose
MongoDB
Processing Live Game Data in Realtime
17. Realtime Live Game Updates
•
Game play requires up-to-the-minute stats from live
events for user scoring
•
These data are stored in JSON format for the app
•
The JSON data has to be updated frequently with
stats and player points
•
Support multiple live games for multiple apps on the
same platform
19. New Way: With MongoDB, we can simply
update the JSON data in the player’s document
db.players.update(
{
_id: ObjectId(“52be0717978ca03fc1984069"),
‘games.g':'2013-e.39141
},
{
$set:{
'games.$.b': "1-for-2: Ground out, Walk, Home run"
},
$inc:{
'points': 36
}
}
)
20. In general, the system with
MongoDB is much simpler,
faster, more scalable
22. Building User Leaderboards
•
Leaderboards updated in realtime
•
96 leaderboards in BUNT
•
Final output is constructed on-demand, no cache
•
User scores are stored in sorted sets in Redis
(ranking is automatic)
•
Redis is an in-RAM key-value data store
25. Scaling Example: Buying
Packs of Cards
•
In order to support complex trading algorithms,
each user profile needs to contain a reference to
the owner’s card collection
26. Initial Structure of User Profile with
Embedded Card Summary Documents
•
Profile contains an embedded
document of card summaries
•
When users buy cards, the
profile can grow out of its
allocated space
•
MongoDB creates a new,
bigger allocation for the user
profile
27. Profiles were refactored to
include only player IDs
•
Finite number of players in the system, so size of
player IDs list is limited
28. Basic Metrics
•
On average, 1 pack sold per second
•
Consistently top 10 grossing sports app
•
Up to 30,000 requests per minute
•
Up to 2,000 OPS
29. Conclusions
•
MongoDB great for apps, especially social
•
JSON-ready data
•
Normal NoSQL arguments
•
For realtime leaderboards, Redis provides simple and
fast “automatic sorting” of user scores
•
Don’t embed documents if you hope for them to grow
•
Easy to learn
Expanded social features—commenting, liking, reporting, admin tools, more in future
People can buy unlimited cards—card store with in app currency, packs of random cards—have to buy a lot of packs to get cards you
Change in game play— start/sit with so many cards
Cards Home: My Cards, Start-Sit Trading Trade Talk Buy Pack
Live game loop
Leaderboards
Card buying
MySQL is needed to store data from the feed provider
Heartbeats read that data and store them in MongoDB
More detail later
Real-time
Done when user makes a call
more later
When a player hits a home run, fan needs to get points asap
Human at feed company enters event data
Data gets pushed to the feed and goes to the MySQL db
David Ortiz hit a HR in Game 2 of the WS
Gets 36 pts for the HR
New box score is XXX
set the box score, increment the season points and game points
One step
Redis is an all-RAM key-value data store
Since redis stores the user IDs and scores in sorted sets, all the ranking is kept up to date whenever it is updated.
When special cards were released into packs, people bought a lot, increasing their cards and creating new profile documents—TIMEOUTS
If this is happening with many users, everything slows down
