4. Utopia Kingdoms
● Fantasy strategy game
● Build your own Kingdom
● Create armies and attack other Kingdoms
● Join other Kingdoms in an Alliance
● Manage resources
● Available in Facebook and Kongregate
http://www.facebook.com/UtopiaKigdomsGame
http://www.kongregate.com/games/JoltOnline/utopia-kingdoms
9. Some points of interest
(will discuss them later)
●
Your resources (population, gold, food, etc)
grows with time
●
You actions (build something, attack a player)
typically takes some time
●
Players are ranked against the rest
●
You can add friends and enemies
11. Measurement tools
● OS tools
● Task manager (top)
●
IO Monitor (iostat)
● Monitoring tools (Munin, Nagios)
● Logs
●
Needs to find a good compromise detailed/relevance
● Profiling
12. You've got to love profiling
●
Generate profiles with cProfile module
Profile whole application with
python -m cProfile -o file.prof my_app.py
(not very useful in a web app)
●
If you're using a framework, profile only your
functions to reduce noise
14. Analyzing profile
●
gprof2dot
●
Using dot, convert to graph
gprof2dot -f pstats file.prof | dot -Tpng -o file.png
●
Good for workflows
●
RunSnakeRun
●
Good for cumulative times
18. All static should be out of python
●
Use a good web server to serve all static
content (Static HTML, CSS, JavaScript code)
●
Some options
●
Apache
●
Nginx
●
Cherokee
●
Amazon S3
20. Example
●
Asking for friends/enemies to DB
●
Costly request in SimpleDB (using SQL statement)
●
On each request
●
Cache the friends on memcache for 1 hour
●
Invalidate the cache if adding/removing
friends or enemies
21. Caching caveats
●
Cache only after knowing there is a problem
●
Do not trust in cache for storage
●
Take a look on size of cached data
●
Choosing a good cache time can be difcult /
Invalidate cache can be complex
●
Some data is too dynamic to be cached
22. Caching is not just memcached
●
More options available:
●
Get on memory on start
●
File cache
●
Cache client side
23. Parse templates just once
●
The template rendering modules have options
to parse the templates just once
●
Be sure to activate it in production
●
In development, you'll most likely want to
parse them each time
●
Same apply to regex, specially complex ones
25. Rankings
●
Sort players on the DB is slow when you grow
the number of players
●
Solution:
●
Independent ranking server (operates just in
memory)
●
Works using binary trees
●
Small Django project, communicates using xmlrpc
●
Inconvenient:
●
Data is not persistent, if the rankings server goes
down, needs time to reconstruct the rankings
26. Database pulling - Resources
●
There was a process just taking care of the
growth of resources.
●
It goes element by element, and increasing the
values
●
It pulls the DB constantly, even when the user has
their values to maximum
●
Increment the resources of a user just the next
time is accessed (by himself or by others)
●
No usage of DB when the user is not in use
●
The request already reads from DB the user
27. Database pulling - Actions
●
Lots of actions are delayed. Recruit a unit,
buildings, raids...
●
A process check each user if an action has to
be done NOW.
●
Tons of reads just to check “not now”
●
Great delay in some actions, as they are not
executed in time
28. Database pulling - Actions
●
Implement a queue to execute the actions at
the proper time:
●
Beanstalk (allows deferred extraction)
●
A process listen to this queue and performs the
action, independently from request servers.
●
The process can be launched in a diferent
machine.
●
Multiple process can extract actions faster.
30. Amazon SimpleDB
●
Key – Value storage
●
Capable of SQL queries
●
Store a dictionary (schemaless, multiple
columns)
●
All the values are strings
●
Access through boto module
●
Pay per use
31. Problems with SimpleDB
●
Lack of control
●
Can't use local copy
– In development, you must access Amazon servers (slow
and costly)
●
Can't backup except manually
●
Can't analyze or change DB (e.g. can't define
indexes)
●
Can't monitor DB
32. Problems with SimpleDB
●
Bad tool support
●
Slow and high variability (especially on SQL
queries)
●
Sometime, the queries just timeout and had to be
repeated.
34. MongoDB
●
NoSQL
●
Schemaless
●
Fast
●
Allow complex queries
●
Retain control (backups, measure queries, etc)
●
Previous experience using it from ChampMan
35. Requisites of the migration
●
Low-level approach
●
Objects are basically dictionaries
●
Be able to save dirty fields (avoid saving
unchanged values)
●
Log queries to measure performance
36. MongoSpell
●
Thin wrap over pymongo
●
Objects are just dictionary-like elements
●
Minimal schema
●
Fast!
●
Able to log queries
●
It will probably be released soon as Open
Source
42. Scalable vs Efcient
Scalable Efficient
●
Can support more ●
Can support more
users adding more users with the same
elements elements
Work on both to achieve your goals