More and more companies provide Web APIs for their core services as an effective way to foster an ecosystem, but you need to have an API platform to host and manage the Web APIs. Building one from scratch can be challenging. This session teaches you how to create your API platform based on oAuth 2.0, REST, and NoSQL technologies by using open source stacks, including Apache projects such as Tomcat, Tuscany, Wink, Amber, and HTTP clients together with NoSQL solutions such as MongoDB and Redis. From real-world experience, you’ll learn the key components and techniques for creating a robust and scalable Web API server.
Unblocking The Main Thread Solving ANRs and Frozen Frames
Building a Web API Platform with Open Source oAuth 2.0, REST, and NoSQL (JavaOne 2012)
1. BUILDING A WEB API
PLATFORM WITH OPEN
SOURCE OAUTH 2.0, REST,
AND NOSQL
SESSION 6946
JAVAONE 2012
2. About the speakers
Raymond Feng
Software Engineer – Shutterfly, Inc.
Staff
Member – Apache Software Foundation
Committer – Apache Tuscany, Wink, Amber
Co-author – Tuscany SCA In Action
3. Agenda
Why a Web API platform
The key components of Web API platform
stack
OAuth 2.0
Http Reverse Proxy
Metrics
API discovering, exploring and playing
Q&A
5. Why a Web APIs platform?
Why Web API?
A great way to build the ecosystem
For some companies, APIs = products
Proliferation of mobile clients
Universal access for internal systems/web or mobile
fronts/third party apps
This talk is about the platform/infrastructure
behind the curtain to support Web API calls.
It’s NOT about Web API
design/development/security.
6. What’s behind the scene?
What’s behind an oAuth 2.0 protected REST
API call?
GET
https://api.<mycompany>.com/me/albums?acces
s_token=<oAuth 2.0 access token>
POST
https://api.<mycompany>.com/me/albums?acces
s_token=<oAuth 2.0 access token>
Content-Type: application/json
Accept: application/json
Request body: {“name”: “Summer 2012”}
7. A Web API Platform
QoS plugins
HTTP Reverse Proxy
Auth Protected
Protected
Resource
Resource
Metrics
Client
Client
Application Throttling
Application
Mediation
User Sign-in
Page
…
Infrastructure
Distributed data grid
(service registry, metrics, quota/usage)
Persistenc
Security e
(Identity management, client application registration, user
authentication, token management, resource ACLs)
11. oAuth 2.0
Based on Apache Amber which implements
the latest oAuth 2.0 spec
http://tools.ietf.org/html/draft-ietf-oauth-v2-31
http://incubator.apache.org/amber/
Additional SPIs are identified for oAuth 2.0
related metadata management and integration
with the security infrastructure
12. oAuth 2.0 concepts
OAuth defines four roles:
resource owner: An entity capable of granting access
to a protected resource (e.g. end-user).
resource server: The server hosting the protected
resources, capable of accepting and responding to
protected resource requests using access tokens.
client: An application making protected resource
requests on behalf of the resource owner and with its
authorization.
authorization server: The server issuing access
tokens to the client after successfully authenticating
the resource owner and obtaining authorization.
13. oAuth 2.0 flows/grant types
An authorization grant is a credential
representing the resource owner's
authorization (to access its protected
resources) used by the client to obtain an
access token.
oAuth 2.0 specification defines four grant
types:
authorization code
implicit
resource owner password credentials
client credentials
14. User cases: Trusted client
applications
Trusted clients include:
Internal applications (running at the server side or
client side)
Mobile clients
Mapping to oAuth 2.0
Clientcredentials for access token (super user)
Resource owner password credentials for access
token (run as the “resource owner”)
15. Use cases: Third party
applications
We plan to roll out web APIs to 3rd party
applications
oAuth 2.0 will be used as the
authentication/authorization mechanism
Mapping to oAuth 2.0
Authorization code flow
Implicit grant flow
17. oAuth 2.0 SPI – resource ACL
Determine if a resource is protected
Public
Client-id
specific
Resource-owner specific
Who are the resource owners?
Scopes of the access
{"scope": "media", {"id": "media_resource",
"description": "Media", "operations": [ "ALL”],
"expiresIn": 604800, "path": "/media",
"requiredAccessLevel": 1, "scopes": [ "media”]
"resourceOwnerAuthorizationRequired": }
false
}
18. oAuth 2.0 SPI – Authentication &
Authorization
Make sure a client is registered
Make sure a token/code is valid
Authenticate a client using
client_id/client_secret
Authenticate a resource owner using user
name/password
Check the token against the protected
resources (ACL)
Establish the principal/subject
19. oAuth 2.0 SPI – Token
Management
Generate access/refresh tokens for a given client,
resource owner, and scopes
Generate authorization codes for a given client,
resource owner, and scopes
Look up the token metadata based on the token
string
Look up the authorization code metadata based
on the code string
Expiration
Refresh tokens
MongoDB as the backend store for tokens
20. oAuth 2.0 infrastructure
Client Client
Authorization Registration Client
Registration
Server Manager Registrations
Endpoint
Resourc
e Owner
Authorization Users
Authenticator
Endpoint (Resource Protected
Owners) Resources
User Internet (such as
Agent user media
(browser) or address
Token
Endpoint Token Tokens and book)
Manager Authorization
Codes
Resource
Client Access
Manager Resource
oAuth 2.0
Permissions
Resource and Scope
Filter Definitions
HTTP
Resource proxy
Server
21. Mapping oAuth 2.0 scopes
Scope
Scope
Scope
Protected
Protected
Resource
oAuth 2.0 Protected
Resource
token GET /services/addressbook/…
Resource
POST/services/addressbook/…
Client
GET /services/media/…
…
…
Resource
Owner
See an example at:
http://developers.facebook.com/docs/reference/api/permissions/
23. Client/Resource management
(administrative)
We need to have UI to
manage the resource endpoints (URI patterns
and HTTP operations)
define oAuth 2.0 scopes to map to a list of
resource access permissions
Manage client applications (enable/disable,
setting quota, …)
Manage access tokens/authorization codes
24. Dashboard
Monitor the API usages (administrative)
By client id
By user id
By resource URIs
By timeline
Let developers see their client applications
(developer)
Registrations
Usages
Granted permissions
25. REST APIs for API
management
Get the list of defined scopes
Display a scope
Get the list of defined resources
Display a token
Display an authorization code
Display a client registration
Get the list of registered clients for a given user
List tokens by client id/user id
List authorization codes by client id/user id
List granted permissions
28. HTTP reverse proxy
DO NOT deploy the services on the api server
directly
API server dispatches API calls to the back-
end services
Open source tools:
Apache http components: http://hc.apache.org/
29. HTTP reverse proxy – connection
management
HTTP connection pool
Persistent connections (keep-alive)
Check the keep alive settings at the backend web
servers
Chunked transfer encoding support
Some Nginx servers don’t support chunked mode
Make sure the http entity is “consumed” so that
the http connection will be released back to the
pool
http://hc.apache.org/httpcomponents-client-
ga/tutorial/html/connmgmt.html
30. Routing/Proxying
URL mapping service registry
URL templating
<uriMapping>
<source>/addressbook/{uid}</source>
<target>http://backend.xyz.com/services/addressbook/{uid}</target>
</uriMapping>
api.xyz.com/addressbook/me/contatcs
The
“me” or “self” will be replaced with the resource
owner from the oAuth 2.0 access token
31. Sync vs. Async
Sync:
Servlet 2.5 and Apache HTTP client 4.x
Async:
Servlet 3.0 async filter (Tomcat 7.0.x or Jetty 8.x)
Apache HttpAsyncClient 4.0 beta 2
final AsyncContext asyncContext =
request.startAsync();
asyncContext.start(new Runnable() {
public void run() {
asyncDispatch(asyncContext, target);
}
});
32. Java vs. Node.js
Which one is better?
Overhead
Scalability
Servlet 2.5 sync mode + Apache http client
Servlet 3.0 async mode + Apache http async
client
Node.js event-driven mode + http
33. Mediations
Some examples:
Protocol/data translation
XML JSON
API tracking
https://github.com/codahale/metrics
http://graphite.wikidot.com/
Analytics
CORS enablement
35. API discovery and
documentation
Some apis/tools that help developers to discover
and explore the apis
A spec to describe the apis (urls, methods,
input/output/exception data model)
A UI to discover and explore the apis
Some integration with the api implementation stacks
such as JAX-RS to introspect the api signatures
Client code generation tool
Open source tools
http://swagger.wordnik.com/
https://developers.google.com/discovery/