1. Building TweetEngine
Ikai Lan
Developer Relations, Google App Engine
ikai.l@google.com
Twitter: @ikai
2. Goals of this talk
Using TweetEngine to
explain key concepts of
building cloud applications
Explaining App Engine as
a development
environment
Whet your appetite!
3. What is App Engine?
App Engine is a platform
You build & test your app
Then upload your app to Google
App Engine runs everything
No need to worry about machines,
network, storage, scalability, etc.
4. The king of ease-of-use
Extremely rapid development
Very low barrier of entry
Simple yet robust syntax
Rich library of
packages/modules
App Engine's first language API
7. OAuth
Secure method for granting
permissions to third-party
integrations
Allows sites to use a trusted
identity provider (Google,
Twitter)
"Valet key" for the internet
Image and "valet key" source: (http://hueniverse.com/oauth/guide/intro/)
8. Standard OAuth flow (User)
What does the User see?
1. Link to login with Twitter
2. User is redirected to an http:
//twitter.com URL
3. User logs in, grants access
4. User is redirected back to site
9. Standard OAuth flow (Server)
What does the server do?
1. Acquire request token
2. Generate login URL
3. Redirect or link user to this
URL
4. After user has logged in,
Twitter will redirect to a
callback URL on server
5. Server saves token passed to
callback URL
10. TweetEngine + OAuth
OAuth request token saved in TwitterAccount model
Saved per Twitter Account
oauth.py handles signing
apis.py handles proxied Twitter API calls
11. Logging in to Twitter using OAuth
# src/tweetengine/add.py
auth_token =
self.request.get("oauth_token")
auth_verifier =
self.request.get("oauth_verifier")
user_info = client.get_user_info(auth_token,
auth_verifier=auth_verifier)
# Create the twitter account account = model.TwitterAccount.
get_or_insert(
user_info["username"],
oauth_token=user_info["token"],
oauth_secret=user_info["secret"],
name=user_info["name"],
picture=user_info["picture"])
13. Twitter API calls
- Web handlers user AJAX
- Backend calls go to TwitterApiHandler
# src/tweetengine/handlers/apis.py
for k,v in params.items():
if isinstance(v, unicode):
params[k] = v.encode('utf8')
# Join all of the params together.
params_str = "&".join(["%s=%s" % (encode(k), encode(params[k]))
for k in sorted(params)])
# Join the entire message together per the OAuth specification.
message = "&".join(["GET" if method == urlfetch.GET else "POST", encode(url),
encode(params_str)])
# Create a HMAC-SHA1 signature of the message.
key = "%s&%s" % (self.consumer_secret, secret)
# Note compulsory "&".
signature = hmac(key, message, sha1)
digest_base64 = signature.digest().encode("base64").strip()
params["oauth_signature"] = digest_base64 return urlencode(params)
14. OAuth
The "de facto" standard for authentication on the web
against a third party API
TweetEngine's support can be easily generalized to be used
on any OAuth supporting site
Highly recommend: New sites, do not build identity system,
use OAuth or OpenID
15. i18n - internationalization
Problem: maintaining
localized versions
Localization is more than
translations
TweetEngine already
supports English, German
and Italian!
Image source: http://www.flickr.
com/photos/natematias/22132919
16. Workflow
1. Developer marks strings as requiring translation
2. Translators work on .PO files
3. These are compiled into efficient .POT files
4. At runtime, translations are looked up when pages rendered
using "Accepts-Header"
17. What does a .po file look like?
#. Default: "Collaborative tweeting"
#: tweetengine/templates/base.pt:36
msgid "subline"
msgstr "Tweeting collaborativo"
#. Default: "Welcome!"
#: tweetengine/templates/index.pt:3
msgid "title-welcome"
msgstr "Benvenuto!"
18. Sample template code
tweetengine/templates/index.pt:
<metal:title fill-slot="title" i18n:translate="title-welcome">Welcome!</metal:title>
tweetengine/templates/base.pt:
<h2><a href="/" i18n:translate="subline">Collaborative tweeting</a></h2>
Run from TweetEngine root: bin/i18nize
19. src/tweetengine/handlers/base.py
# Instantiate Chameleon template loader
from chameleon.zpt.loader import TemplateLoader
from tweetengine import model
from tweetengine.menu import mainmenu
tpl_path = os.path.join(os.path.dirname(__file__), "..", "templates")
tpl_loader = TemplateLoader(tpl_path)
# Later on in the file
template_vars['target_language'] = self.request.headers.get('Accept-Language', None)
tpl = tpl_loader.load('base.pt')
template_vars['master'] = tpl.macros['master']
tpl = tpl_loader.load('macros.pt')
template_vars['macros'] = tpl.macros
tpl = tpl_loader.load(template_file)
self.response.out.write(tpl(**template_vars))
20. Remember: i18n > translations
Resizing UI elements for languages with long/short words
Right-to-left languages (Hebrew, Arabic) and layouts
Time and date formats
Metric/English system
Much more ... this just a place to start!
21. App Engine is also about tooling
Image source: http://www.flickr.com/photos/sparr0/4584964212/
22. Two App Engine tools
AppStats Task Queues
Application profile that is Background queueing
App Engine aware and worker mechanism
Image source: http://www.flickr.com/photos/spbatt/3928384579/
23. AppStats
Track expensive requests
Expose bottlenecks in your
code
Show you what code is
being executed the most
for "hotspot optimization"
Run in production with
minimal overhead
26. How does it work?
Uses Call Hooks
pre‐call hook records start .me, request, stack
post‐call hook records end .me, mcycles, response
Measuring times:
real time using Python’s wall clock API
API megacycles using rpc.cpu_usage_mcycles
CPU megacycles using quota.get_request_cpu_usage()
Get stack contents from Python’s sys._getframe()
Recording accumulates in memory object
Written to memcache at end of each request
30. Task Queues in App Engine
# Creating a task # src/tweetengine/model.py taskqueue.Task
(eta=send_time, name=task_name, url=twitter.
ScheduledTweetHandler.URL_PATH).add()
# Task Handler
# src/tweetengine/handlers/twitter.py
class ScheduledTweetHandler(webapp.RequestHandler):
def post(self):
publishApprovedTweets()
def publishApprovedTweets():
# Iterate through unsent OutgoingTweets and send them
31. Task Queues in TweetEngine
Used to schedule Tweets
ETA parameter used to set an approximate execution time
Tasks are named - this allows us to set times to check for
OutgoingTweets that should be sent at this time and not
duplicate work (we don't have to use cron)
32. Task Queues
Allow developers to build asynchronous workers without
hassle of building out queueing/polling system
Handles task uniqueness and transactional task creation
Can be scheduled, throttled
33. Summary: TweetEngine technologies
1. OAuth - "Valet key" for internet applications
2. i18n - Using templates and message bundles in web
applications
3. AppStats - App Engine request profiling toolkit
4. Task Queues - Background work for App Engine
applications
