Rapid web development using tornado web and mongodb
1. Rapid Web Development with
Tornado Web and MongoDB
Ikai Lan
Twitter: @ikai
Friday, June 10, 2011
2. About the speaker
• Developer Relations at Google
• Software Engineering background
• Lives in San Francisco, CA
• Writes Java, Python on a day to day basis
• Twitter: @ikai
Friday, June 10, 2011
3. This talk
• Why Tornado Web and MongoDB?
• Whirlwind tour of Tornado (get it?!)
• Intro to MongoDB
• Brief look at a demo app
Friday, June 10, 2011
4. What I won’t be talking
about
• Scalability - loaded topic
• Reliability - another loaded topic
Friday, June 10, 2011
9. So really, my talk should
have been called “The
Joy of Tornado Web and
Mongo DB”
Friday, June 10, 2011
10. Intangibles
• Tornado - fast development, very few rules
• MongoDB - freedom from a predefined
schema
• Python, of course. Dynamically typed
variables, duck typing plus everything else
that is cool about the language
Friday, June 10, 2011
11. Tornado Web history
• Open Sourced by Facebook
• ... formerly of FriendFeed
• ... former Googlers (that’s why the
framework looks like App Engine’s webapp
- they wrote it)
• Tornado Web powers FriendFeed
Friday, June 10, 2011
13. Tornado features
• Out of the box auth with Twitter,
Facebook, Google and FriendFeed
• Out of box support for localized strings
• Simple templating that allows arbitrary
Python code
• Asynchronous requests
• Small codebase
Friday, June 10, 2011
14. Simple handlers
import tornado.ioloop
import tornado.web
class HelloHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world :)")
class GoodbyeHandler(tornado.web.RequestHandler):
def get(self):
self.render("goodbye.html",
message=”Goodbye, world”)
application = tornado.web.Application([
(r"/hello", HelloHandler),
(r"/goodbye", GoodbyeHandler),
])
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
Friday, June 10, 2011
15. Asynchronous nature
class MainHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
def get(self):
http = tornado.httpclient.AsyncHTTPClient()
http.fetch("http://friendfeed-api.com/v2/feed/bret",
callback=self.on_response)
def on_response(self, response):
if response.error: raise tornado.web.HTTPError(500)
json = tornado.escape.json_decode(response.body)
self.write("Fetched " + str(len(json["entries"])) + " entries "
"from the FriendFeed API")
self.finish()
Friday, June 10, 2011
16. Unrestrictive templating
{% for student in [p for p in people if p.student and p.age > 23] %}
<li>{{ escape(student.name) }}</li>
{% end %}
# Sample of arbitrary functions in templates
def add(x, y):
return x + y
template.execute(add=add)
### The template
{{ add(1, 2) }}
Friday, June 10, 2011
19. MongoDB is an object
database
• No schema - anything that can be a JSON
object can be stored
• You can define new collections on the fly
Friday, June 10, 2011
20. Getting started with
Mongo in 4 steps
• Download a binary for your system
• Create a data directory (mkdir -p /data/db)
• Run mongod
• Install pymongo
• ... start writing code!
Friday, June 10, 2011
21. Pymongo code sample
connection = pymongo.Connection()
database = connection["your_database"]
def get_or_create_location_by_id(location_id):
"""
Attempts to fetch location data from database. If it doesn't exist,
create it. Note that this is NOT concurrency safe.
"""
location_data = database["locations"].find_one({ "_id" : location_id })
if location_data is None:
location_data = { "_id" : location_id,
"guards": [],
"owner" : None,
"history" : [],
"last_extort_time" : None
}
database.location.save(location_data, safe=True)
return location_data
Friday, June 10, 2011
22. Pymongo queries
# Add Tyson as a guard to every location owned by “Joe Smith”
locations = database["locations"].find({ "owner" : "Joe Smith" })
for location in locations:
location["guards"].append("Tyson")
database["locations"].save(location)
# Find everyone who has Tyson as a guard
locations = database["locations"].find({"guards" : "Tyson"})
# Find everyone who has *ONLY* Tyson as a guard
locations = database["locations"].find({"guards" : ["Tyson"]}) # Note []s
# Find everyone who has Tyson as a guard whose owner is “Ikai Lan”
locations = database["locations"].find({"guards" : "Tyson",
"owner" : "Ikai Lan" })
Friday, June 10, 2011
24. Freedom from
• ... waiting to get started
• ... predefining a schema
• ... worrying about complex data structures
that don’t fit well into the SQL box.
Anything that is JSON can be stored!
Friday, June 10, 2011
25. More MongoDB
features
• Simple Geospatial indexing
• GridFS - distributed filesystem running on
MongoDB
• Out of the box mapreduce
• Out of the box replication, data partitioning
Friday, June 10, 2011
26. Putting it all together
• MobSquare, a location based game that
uses the Facebook API.
• Mashup: Mafia Wars + FourSquare
• https://github.com/ikai/mobsquare-demo
• “Check in” to locations, take them over,
extort money and fight other gangs
Friday, June 10, 2011
27. Why are Tornado/
Mongo a fit?
• If I haven’t said it enough yet, they’re fun
• Facebook API performance varies -
asynchronous so we can serve requests
while waiting
• Highly structured player data
Friday, June 10, 2011
28. OAuth 2 upgrade flow
class OnLoginHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
def get(self):
# Store this somewhere
code = self.get_argument("code")
access_token_url = ACCESS_TOKEN_URL_TPL + code
client = httpclient.AsyncHTTPClient()
client.fetch(access_token_url, self.on_fetched_token)
def on_fetched_token(self, response):
""" Callback inokved when the auth_token is fetched """
matches = ACCESS_TOKEN_REGEX.search(response.body)
if matches:
access_token = matches.group(1)
client = httpclient.AsyncHTTPClient()
# lambda is effectively a function factory for us
client.fetch(API["profile"] % access_token,
lambda response: self.on_profile_fetch(response, access_token))
def on_profile_fetch(self, response, access_token):
""" Callback invoked when we have fetched the user's profile """
profile = json.loads(response.body)
profile["access_token"] = access_token
profile_id = db.save_profile(profile)
self.set_secure_cookie("user_id", str(profile_id))
self.redirect("/") # implictly calls self.finish()
Friday, June 10, 2011
29. Known gotchas
• Immaturity of frameworks, tools
• Tornado templating errors result in
somewhat useless stack traces
• MongoDB nuances
• Tornado community fairly small
Friday, June 10, 2011
30. Questions?
• Ikai Lan - @ikai on Twitter
• Github: https://github.com/ikai
• Google Profile: https://profiles.google.com/
ikai.lan/about
• Thank you for having me at Pycon!
Friday, June 10, 2011
31. Attributions
• Slide 6: Kirstin Jennings - “big smile!” http://www.flickr.com/photos/methyl_lives/2973265796/
• Slide 8: Tony Blay - “I am a bird now!” http://www.flickr.com/photos/toniblay/59415205/
• Slide 12: Antonio Sofi - “whirlwind” http://www.flickr.com/photos/webgol/36546734
• Slide 17: Tornado Web documentation - http://www.tornadoweb.org/documentation
• Slide 18: Christine of Robot Brainz - “Cassette” http://www.flickr.com/photos/robotbrainz/
2786347158/
• Slide 23: Samuel Stroube - “Play-Doh on a Picnic Table” http://www.flickr.com/photos/samoube/
203586664
Friday, June 10, 2011
