Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/WJraOI.
Paul Hill presents a case study of building an API with a short deadline using Node.js, WebSocket, MongoDB, JSON, Promises, Swagger, Memcached, Varnish and Hypermedia ReST. Filmed at qconnewyork.com.
Paul Hill is a software architect, designer and developer at KIXEYE, and resident expert on all things ReST and Hypermedia. An enterprise Java developer for most of his career he is now very much a polyglot paratrooper, spending most of his time designing and building APIs with Node.js to provide delightful features to KIXEYE's online competitive gaming community.
1. ALL YOUR API
ARE BELONG TO US
QCon New York 2014Everything as a servicePaul Hill
2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/kixeye-rest-api
3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
6. The Enrichment Center regrets to inform you
that this next test is impossible.
Make no attempt to solve it.
The Goal
★ Social graph search
★ Chat
★ Identity (incl. game aliases)
★ Player profiles
★ Friends and following
★ Leaderboards
★ Messaging
★ Avatars
★ Live marketing channel
★ Online presence
★ Player search
★ User account management
★ Real-time player game stats
★ Community management (incl. bans)
★ Groups & Alliances
★ Authentication
○ OAuth2
○ Facebook Connect
○ Google Auth
★ Game launch canvas
★ Geo-location
★ Name validation
★ Email validation
★ Closed beta invite control
★ Newsfeed
★ Gifting
★ Forums
★ Payments
https://api.kixeye.com/api/v2/docs/
7. Fear not! I will inspire you all by charging
blindly on!
Tail Winds
★ Small team of Rockstar Developers
★ Positive culture + optimism
★ Given a goal, not direction
★ Known traffic patterns
○ ~30MM active players
○ ~120k CCU playing games
○ ~10k - 30k CCU active on web
property
★ User experience trumps consistency
★ Time to market trumps cost
★ Over provisioning
8. The right man in the wrong place can make all
the difference in the world.
The Good
★ ReST
★ JSON Schema v4
○ Validation
○ zSchema
★ Swagger
○ Live interactive docs
★ Mongo
○ easy JSON storage
★ Promises
○ Error handling guarantees
○ Bluebird
★ Elasticsearch
○ use strict schema
○ ngrams
○ DisMax
★ Measure all the things!
○ Graphite & StatsD
○ ready fire fire fire...
★ Websockets
○ SockJS
★ Easy to grok, fork and patch repos in github
★ CLS (continuation local storage)
○ performance plus
○ encourages reusable functions
★ Automated unit testing by response code
★ 3 tiers (Transport, Logic, Persistence)
★ URI based caching
○ Varnish
○ Memcached
○ focus on problem #1
12. War, war never changes.
The Bad
★ Compressed timelines
★ Dynamic requirements
★ Mongo
○ unique constraints limited on
sharded collections
○ access pattern outgrew DDS
○ storing relationships
○ write scaling
○ background indexing
★ Load balancing socket.io
★ Performance of overhead of Q
★ V8
○ gc
○ Poor live profiling tooling
★ Excessive calls from builds to GitHub
★ npm
○ flakey, requires a mirror
★ CLS
○ requires a lot of shims or
context is lost
★ Jasmine
○ fails and just keeps on going
○ beware the async setup collision
13. I am the vanguard of your destruction.
The Ugly
★ Natural key based load balancing
(The Aggregator)
★ Back pressure
★ Memory leaks
○ Thrift JS - oneway void - Flume
○ Default error handling in libs
○ Q long stack traces
★ Insane default configuration in libs
★ Supernodes
○ $in
★ Reactive cascade failure
○ Default retry
○ true exponential back off with
no cap
○ Node Zookeeper heartbeat
timeout
○ gc behaviour under stress
○ load rebalancing