RESTful web application development with Java and Kauri - presented @ Devoxx University 2008

1. www.devoxx.com

2. REST In Peace:
Javanian RESTfulness
Steven Noels
Marc Portier
www.devoxx.com s

3. Overall Presentation Goal
Learn how to architect, prototype and build
RESTful Web Services and Interactive Web Sites
using appropriate Java frameworks.
www.devoxx.com s

4. Speaker’s qualifications
Steven and Marc combine 25+ years of Java and XML-
based web development and co-founded Outerthought.
A involved history of application development with Apache
Cocoon.
Outerthought has been building an open source Content
Management System that follows REST-principles named
“Daisy” for the past 4+ years.
Outerthought is currently building a REST-supporting
development framework: “Kauri”.
We are an early REST adopter as we were with Java in
1996.
4
www.devoxx.com s

5. Overall Presentation Goal
Learn how to architect, prototype and build
RESTful Web Services and Interactive Web Sites
using appropriate Java frameworks.
www.devoxx.com

6. Underlying theme
• Help!
My boss wants me to develop REST applications?
www.devoxx.com s

7. What REST means ....
A pragmatical, no-nonsense approach and guiding set of
ideas.
A way to think about Web Services!
• As in: ”Distributed Computing System Architectures”
• Sometimes leading to almost religious debates
A nostalgic 'return to the web movement'
Probably a prophecy about the Semantic Web
www.devoxx.com s

8. The problem with REST
You can't just buy it.
(tools, products,...)
Big words, theoretic, even
esoteric guidance
Often negative advise:
• Do not do this or that.
Too little concrete tips, steps to follow, ...
An architectural style – not an architecture
• Lack of building plans
www.devoxx.com s

9. Resource Oriented Architectures
aka ROA
Term coined in the RWS Book
More practical advise and groundwork
For
• A REST-style architecture
• For building services
• Tied to the Web
• And its main HTTP protocol
www.devoxx.com m

10. Web Essence
Q1: Method Information
• How does the client convey its intentions to the server?
• HTTP method: GET, PUT, POST, DELETE
• URI: e.g. GET /path?method=removeThat
• Entity Body: Envelope with certain format (e.g. SOAP)
Q2: Scope Information
• How does the client tell the server which (part of the) data to operate on
• URI (path or request params)
• Entity Body: Envelope with certain format (e.g. SOAP)
www.devoxx.com m

11. Different types of Web Services
I. RESTFul/Resource Oriented
• Method info in HTTP Method.
• Scoping info in URI.
II. RPC Style
• Entity Body holds Method and Scoping info.
• Single URI: the 'end-point' | Single Method: POST
III. Hybrid
• Method info in the URI, next to the scope info.
• Difficult to spot, only visible when not read-only.
• Often grown out of classic websites.
www.devoxx.com m

12. ROA Terminology
#1. Resources ~1. Addressability
#2. URI's ~2. Statelessness
#3. Representations ~3. Connectedness
#4. Links Between them ~4. Uniform Interface
The four Concepts The four Properties
* as proposed by the RWS Book
www.devoxx.com m

13. #1. A Resource
Something.
Anything.
• Idea, Concept
• Data record, Result, Answer, File,
• Physical item, Real world object.
As soon as it is important enough to
• Reference it by itself.
• Talk about it.
• Get representations from it.
• Perform operations on it.
www.devoxx.com s

14. Kinds Of Web Resources
www.devoxx.com

15. Kinds Of Web Resources
End-User Data – or 'library resources'
• Content (CMS like)
• Binary Downloads and 'attachments'
• Full Text Search-result-lists, linking to the above
• Almost directly consumed by End-Users
“Faceless” - or 'service resources'
• Manageable Data (json or XML) Items (CRUD)
• Query results – Algorithm results (GET only)
• Predefined one-off views/indexes (GET only)
• Consumed by Custom Applications
www.devoxx.com s

16. Kinds Of Web Resources II
Browser-Technology-Feeds – or 'site resources'
• Pages/Page Impressions
• (possibly also aggregated)
• Styling/Skinning (CSS / Images)
• Functional Applications
• Forms
• JavaScript logic (or other RIA approach)
• Consumed by web-browsers,
supporting the end-user experience.
www.devoxx.com s

17. #2. URI
Universal Resource Identifier
It is the name
And the address of a resource
The existential property of the resource
• If it doesn't have a URI, it isn't a resource
• All resources have (at least) one
http://www.w3.org/DesignIssues/Axioms (TBL)
www.devoxx.com m

18. Good URI's Are...
Descriptive
• Reading the URI gives a good idea what to expect.
No surprise / Natural
Following some pattern / structure (Hackable)
• Similar resources have similar URI's, eg
• http://library.org/search/REST
• http://library.org/articles-about/SOAP
Long lasting associated to the resource.
www.devoxx.com m

19. URI to Resource Relation
Can two resources be the same?
Can two resources share the same URI?
• No.
• URI's are UNIVERSAL
(one for the resource in the whole universe)
• Still at any given time two distinct resources might point to the
same data:
• http://project.org/releases/1.0.7.tar.gz
• http://project.org/releases/latest.tar.gz
• Note: two distinct concepts!
www.devoxx.com m

20. URI to Resource Relation II
Can a URI be associated to another resource?
• No. The meaning should stick.
• Note though: Resources do have a live-cycle. They
exist/don't-exist at any given time.
Can one resource have more then one URI?
• Yes. Alternatives make resource more accessible. But
also dilutes the value of each one.
• Relates to: Are my car-keys in my normal spot(s)
• Make sure there is a canonical variant...
www.devoxx.com m

21. Canonical URI's in Practice
Advised when 2 or more URI's exist for one resource
Solution 1:
• All URI's return 200 OK, and representation in body
• Together with special header:
Content-Location: canonical URI
Solution 2:
• Only one URI serves the content (with 200 OK)
• All others return 303 See Also
• Together with the header:
Location: canonical URI
www.devoxx.com m

22. ~1. Addressability
Question: How many URIs should you foresee?
• Answer: more!
• Increase the surface area of your application.
Preference: URI or Description?
1) Goto homepage,
2) Then query for “criteria”
rd
3) Take 3 option...
Random Access Coordinates To All Resources
• URI's to resources are no accidents,
but part of the design. www.devoxx.com m

23. Addressability Matters
URIs can be published anywhere (email, paper, ...)
Addresses can be talked about (link love!)
• Bookmark- and tagging services.
True (addressable) Resources get Web Benefits:
• Caching
• Chaining & Aggregation
• Translation & Validation Services
• Scraping and Mash-up
www.devoxx.com m

24. ~2. Statelessness
Every HTTP request for a resource should happen in
complete isolation.
• Server never relies on information from previous
messages.
Logic Premises:
• Resources are pieces of info on a server
• Addressability says they all should have URI's
Then Statelessness means all server states have a distinct
URI.
• e.g. page 2 from Google results about 'REST'
www.devoxx.com s

25. Statelessness Matters
Superlative of Addressability
• Bookmark-able hooks into earlier conversations
• The 'boemboemboem' without the 'blablabla'
All benefits of addressability,
Now for intermediate states:
• Caching, mash-up, aggregation, translation, direct
referencing, ...
Most importantly: positive effect on scalability
• Easier distribution across load-balanced servers.
www.devoxx.com m

26. Kinds of State
The bad kind: “Conversational State”
• Builds up a conversation/session.
• Introduces dependency across requests.
The good kind: “Application State”
• Cleanly kept up in the client, the process, the end-user
interaction
• Grown by user interaction and received links.
The ugly (hard) kind: “Resource State”
• Server side. Conform the resource life-cycle.
www.devoxx.com m

27. #3. Representations
Plato-nic view on things
Resource = the concept, idea, or principle
Representation = what you really get
• The data, The bytes
• a COPY
• Physical resources often have disappointing
representations (often status info or metadata)
Note: representations can travel upstream.
• Write operations via PUT
www.devoxx.com s

28. Representation Variants
Representations have specific format and language (the resource
often has not)
Clients can 'negotiate' preferred variant:
• Request headers:
Accept: (mime-types)
Accept-language: (iso codes)
Server can describe in which ways representations can vary:
• Response header:
Vary: Accept, Accept-language
• Important for caches!
Vary: * //can not be cached
www.devoxx.com s

29. On Content Negotiation
1 URI per resource is Good™
1 URI per representation is practical
• More visible and clear
• Still a canonical negotiation URI is possible
• Increased “Addressability”
• Passing URI to translation and validation services
Options
• Through request parameter (@google: ?hl=nl )
• In the /path/resource-name.nl.html
www.devoxx.com m

30. #4. Links
Ehrm? Not URI's?
Link = fact that the URI gets encoded in the representation
Wow. WOW! (Let it sip through!)
(Imagine being 12 and )
Getting an SMS with the number
of a new buddy to be SMS-ing with...
Note: soft-referencing by nature.
www.devoxx.com s+m

31. ~3. Connectedness
URI = name-addressing scheme
• Then designing to use them is to ensure Addressability
Links = having representations hold URIs
• Then designing to use them wisely is to ensure
Connectedness.
The end-goal is to achieve self-learning behaviour upon the
clients
• Publish only one entry URI
• cfr. REST-paper: Hypermedia as the engine of
application state
www.devoxx.com m+s

32. ~4. The uniform interface
Limited set of resource interactions
• GET, PUT, DELETE (allow for crud-like operations)
• POST
• The escape for the pragmatics
• The Danger zone (breaking ROA properties)
• HEAD: same as GET without the payload
• OPTIONS: self-description of the resource
• Underused.
•Answer in response header:
Allow: GET HEAD PUT DELETE
www.devoxx.com s

33. The uniform Interface
The various methods are expected to work in a certain
way (Reason: resilience to unreliable networks)
• GET (and HEAD) should be SAFE
• No changes to server-state.
• No side-effects intended by the client.
• Mild side-effects accepted (log, counters, ...)
• PUT and DELETE should be IDEMPOTENT
• Replay/Duplicate of the message does not change
the server-state again.
www.devoxx.com m

34. The uniform Interface
• POST, no restrictions. The Sky is the limit
• Design and document for safety and idempotence!
• e.g. uuid for operation, optimistic locking, ...
www.devoxx.com m

35. More Uniform Interfacing!
NOT only about the request-method
• Response Code! (no error-code hiding please!)
• Request Headers (e.g. content negotiation)
• Response Headers
• Mainly Location, Content-Location
• Don't overdo it though: stay away of subtle nuances
and unused/unknown concepts
Essence: Principle of least surprise.
• So avoid custom X-Headers and cookies
www.devoxx.com m

36. Caching
Caching is a solution for
• Latency and network traffic.
Critical success factor of the web.
• Don't fight it for 'control' of your website.
• Don't see it as a danger or problem.
The opportunity in “stale”
• Design to help caches do what you think is best
• Buzzphrase: stale and scale
www.devoxx.com s

37. Recap: Resource Oriented Architectures
#1. Resources ~1. Addressability
#2. URI's ~2. Statelessness
#3. Representations ~3. Connectedness
#4. Links Between them ~4. Uniform Interface
The four Concepts The four Properties
* as proposed by the RWS Book
www.devoxx.com m

38. Recap: Why REST?
•Increased contact surface and deep linking
•Scaling out over stateless servers.
•Safe operations over unreliable networks.
•Benefitting from the web delivery infrastructure.
•A more balanced distribution of code and responsabilities
across client and server
•Cut the crap: less layers for the same (no WS-*)
•Blending machine and human Internets:
•One filosophy for building web-sites and -services
www.devoxx.com s

39. Going practical
RESTful webapp development
www.devoxx.com s

40. Analysis and design
Questions to be asked
• what are my resources and their URIs
• mapping services/methods to resources
• optionally: UI – screens and flow
• managing application state
Things to build
• a service: API (i.e. REST), runtime environment,
persistency
• a client: custom or browser-based
www.devoxx.com s

41. RESTful web applications
The (web/browser) client needs all the usual stuff
• templates for dynamic behaviour
• forms (!) for interactive behaviour
• skinning
• navigation
• aggregation
Kauri offers all this RIA stuff and more
www.devoxx.com s

42. From entities to resources
1. Define the data-set.
2. Split the data-set into resources.
For each resource
3.Name the resource (URI)
4.Supported request methods.
5.Design exchanged representations
1.Integrate (link) the resources
2.Consider typical/normal flow and interaction.
6.Handle error-conditions and
7.Decide upon error-codes.
www.devoxx.com m

43. From object models to resources
•OK, so persistent objects/CRUD are easy
/data/{entity}[?q=] GET > list
/data/{entity} POST > create
/data/{entity}/{id} GET mani-
PUT > pul-
DELETE ate
•But what about the hard stuff?
● Service methods
● Transactions (locks)
● Sessions
www.devoxx.com s

44. Acceptable POSTs: the obvious
Creating subordinate resources
• when resource location is under client control
• request = PUT + representation
• response = 200 OK
• when server decides on URI
• request = POST + representation
• response = 201 Created + location header
www.devoxx.com m

45. Less obvious POSTs
Appending resource state (e.g. log entries, audit, chat)
• i.o. to avoid full rewrite of resource state
• PUT should be safe, i.e. cannot be incremental
Be pragmatic:
• POST uri?method=put|delete
• Batch of PUTs
Avoid: anything that will introduce the RPC style
• one endpoint URI
www.devoxx.com m

46. Sessions and State
NOTE: Statelessness means the client is in control
When the statelessness theory ends: unreliable,
untrustworthy clients
• e.g. Temptation to go beyond agreed service contract
(size, speed, ...)
Keeping state on the server is hard
• Solution1: State Replication
• Solution2: Session Affinity
• Both conflict with scalability needs (clusters)
www.devoxx.com m

47. About cookies...
No problem! (Like custom X-Request-Headers)
The kind of state they contain is the issue!
• Session-id? BOO! (== conversation state)
• application state = OK. But why not use the URI?
Cookies introduce 'special/non standard' use.
Less 'uniform' interface. More surprise.
Breaks back-button behaviour.
www.devoxx.com m

48. Pragmatic with state
Temporary Resources
• Resources are acceptable server-state containers.
• Side-effect creation is acceptable.
• Lease-expire deletion is acceptable.
• Loading resources in memory (cache) is acceptable (cfr.
LinkedIn)
Distributed Server State is still an (ugly) issue.
• Server-state == Resource State (hence persistent).
• Resource updates must be known across the cluster!
• Solution easier if non-real-time allowed. (latency)
www.devoxx.com m

49. Temporary Resources
Create the resource
POST /temp/upload HTTP/1.1 201 Created
Host: example.com Location: /temp/upload/223
Put subordinate resources in there
PUT /temp/upload/223/resume/cv-mpo.pdf HTTP/1.1
Host: example.com
PUT /temp/upload/223/mugshot/mpo.gif HTTP/1.1
<data> Host: example.com
<data>
Client should keep references
PUT /application/20080923/mpo HTTP/1.1
Host: example.com
{name:”Marc Portier”,
mugshot: “/temp/upload/223/mugshot/mpo.gif”,
resume: “/temp/upload/223/resume/cv-mpo.pdf” }
www.devoxx.com m

50. Transactions
Atomic operations across resources?
Insanely difficult in distributed environments
• In principle HTTP is miles away from XA.
• Practical solutions in relaxing your expectations.
• Still a single server transaction should be achievable.
Old Style:
• Escape into POST: batch-handling
• Request scope matches transaction.
The smart trick:
• Call it a resource.
www.devoxx.com m

51. Transaction Resources
Create the transaction
POST /tx/transfer HTTP/1.1 201 Created
Host: example.com Location: /tx/transfer/738
Put resources in the transaction
PUT /tx/transfer/738/account/938-883 HTTP/1.1
Host: example.com
PUT /tx/transfer/738/account/553-213 HTTP/1.1
balance=300 Host: example.com
balance=710
Put the transaction
PUT /tx/transfer/738 HTTP/1.1
Host: example.com
committed=true
www.devoxx.com m

52. Queueing
Caches work well on the read-end
• “Read Cache” Proxy
On the upstream side
• “Write Accept” Proxy
• 202 Accepted
• Client request not handled in real time.
• Location: can point to resource for status-checking
• Only fights latency. Spread load over time.
• Use cases: classic media driven “vote now” m
www.devoxx.com

53. Onto tooling
We've got all issues covered.
Except the real one:
How to do it?
s
www.devoxx.com

54. Lightweight REST Framework for Java
Clear API : ROA concepts
• Request-Response
• Representations/Entities
• Resource and Reference
Symmetric client-server use!
Routing with parsed URI Templates
Many modules under development, a.o.
• GWT
• JAX-RS
m
www.devoxx.com

55. Website
• http://www.restlet.org
Developer Wiki & Documentation
• http://wiki.restlet.org
Mailing List
• http://restlet.tigris.org/servlets/ProjectMailingListList
SVN
• http://restlet.tigris.org/source/browse/restlet/
Issues www.devoxx.com
m

56. JAX-RS
Annotations for Java coded 'Resource'-POJOS
• Uses URITemplates for mapping
JSR-311
• http://jcp.org/en/jsr/detail?id=311
• Released Oct. 10th 2008
Reference Implementation in Jersey
• https://jersey.dev.java.net/, version 1.0.1 (Dec 1st)
• Subproject of Glassfish
m
www.devoxx.com

57. Sample
package com.mycompany.module1;
import javax.ws.rs.Path;
import javax.ws.rs.GET;
import javax.ws.rs.ProduceMime;
import javax.ws.rs.PathParam;
@Path(quot;helloworld-from-java/{name}quot;)
public class HelloWorldResource {
@GET
@ProduceMime(quot;text/htmlquot;)
public String foo(@PathParam(quot;namequot;) String name) {
return quot;<html><body>Hello <b>quot; + name +
quot;</b> Here is my fully quot; +
quot;qualified class name: quot; +
getClass().getName() + quot;</body></html>quot;;
}
}
m
www.devoxx.com

58. More JAXRS @ Devoxx
Thursday 17:50-18:50
Room 5
by Paul Sandoz
s
www.devoxx.com

59. BREAK
s
www.devoxx.com

60. On your marks...
We've got all issues covered.
Except the real one:
How to do it with Kauri?
s
www.devoxx.com

61. Setting the context
BBC Live Services
Social Services back-end for high-impact websites
• tags, annotations, ratings, votes
• to be called from a PHP front-end
The challenges
• MUST: REST
• MUST: XML
• MUST: scale (load, volume, availability)
s
www.devoxx.com

62. Expected load
Requests Requests Peak Peak Offpeak Offpeak Offpeak Total Total Caching
Minute Hour Hours Requests Share Hours Requests Daily Monthly
(m)
50 3000 2 6000 20% 22 13200 19200 0.576 queue
100 6000 2 12000 20% 22 26400 38400 1.152
500 30000 1 30000 20% 23 138000 168000 5.040 queue
200 12000 2 24000 20% 22 52800 76800 2.304 short
500 30000 2 60000 20% 22 132000 192000 5.760 long
50 3000 1 3000 20% 23 13800 16800 0.504 short
500 30000 2 60000 10% 22 66000 126000 3.780 long
1 60 2 120 10% 22 132 252 0.008 long
100 6000 2 12000 20% 22 26400 38400 1.152 queue
1 60 2 120 20% 22 264 384 0.012
500 30000 2 60000 20% 22 132000 192000 5.760 medium
10 600 2 1200 20% 22 2640 3840 0.115 medium
m 500 30000 2 60000 20% 22 132000 192000 5.760 long
m 10 600 2 1200 20% 22 2640 3840 0.115 long
Total Requests/Month 32.037
s
www.devoxx.com

63. Architecture
s
www.devoxx.com

64. Weight-watcher version
A simple annotation service
s
www.devoxx.com

65. How to start?
Install Kauri
• Java 5+
• Maven 2.0.9
• Kauri (0.3)
Get Kauri from www.kauriproject.org
Deploy Kauri libraries to local development repository (M2)
s
www.devoxx.com

66. Kauri schematics
Kauri runtime + Maven 2 + Spring + bunch of modules
s
www.devoxx.com

67. Supplied Kauri modules
routing
templating
forms
archetype(-prototyping)
dbresources (JPA)
dbmock (json as a data tier for prototyping)
representationbuilder
forms
jQuery
blueprint css
s
www.devoxx.com

68. Why Maven (2) ?
dependency management
build system
repository layout (jars + naming convention)
useful @ dev-time, but also:
alternative for .war packaging @run-time
m
www.devoxx.com

69. DEMO
[1a.] Build hello-world from
maven-archetype
s
www.devoxx.com

70. What have we done:
Run archetype
~/local/kauri/bin/kauri-project-template.sh
Provide a Page, RUN & Test
• serving static resources
• JaxRS + representation + simple template
s
www.devoxx.com

71. kauri.xml
deployment descriptor
• modules used in project
• java & rest service wiring
m
www.devoxx.com

72. Kauri modules
deployed as a JAR in a Maven repo layout
• i.e. unit of reuse – self-contained component
• as opposed to WAR:
• no jar layout
• only reuse possible at the app level
offers Java and/or REST services
m
www.devoxx.com

73. DEMO
[1b.] Give our first module a
more meaningful name and see
how Eclipse development
works.
s
www.devoxx.com

74. Steps
new general project
mvn eclipse:eclipse & import
search for module1
rename all matches to something else
refactor / move using Eclipse
run with -s option for live modifications
s
www.devoxx.com

75. DEMO
[1c.] Templating, routing and
REST service wiring.
s
www.devoxx.com

76. Steps
Add logo route to router.groovy
Change 'mountpoint' in kauri.xml and change links to use
publicUri service
• module can be mounted anywhere now
Add 'search' (rest-)service
• using Google search as a REST service
• with request param passing in router.groovy
s
www.devoxx.com

77. www.devoxx.com

78. About services and URIs
kauri.xml mounts services in the (public) URI space
• use relative (local) URIs !
• rest services have a known interface:
• method support, request params
• returned formats
• content negotiation ...
Within Kauri, rest-services are Restlets
Globally: anything URI accessible (http(s)://, file:// ...
various Restlet goodies ...)
m
www.devoxx.com

79. DEMO
[2.] Build a RESTful data
service via JPA annotated
persistent objects.
m
www.devoxx.com

80. Steps
Use JPA service (dbresources) to create rest-service
/data/annotation GET > list
/data/annotation POST > create
/data/annotation/{id} GET, PUT, DELETE > manipulation
Annotation resource class – create PO
id, URI, text, creation date
add mapping in kauri.xml
test and manipulate with Poster (Firefox extension)
just a web-service, no site yet
m
www.devoxx.com

81. www.devoxx.com

82. DEMO
[3.] Make it real: the simple
annotation tool.
m
www.devoxx.com

83. Weight-watcher version
A simple annotation service
m
www.devoxx.com

84. The feed
m
www.devoxx.com

85. DEMO
[3.] Build the simple annotation
tool.
m
www.devoxx.com

86. Steps
Steven, teh friendly html ui wizard joins me with his USB
stickie
Demo of his work
• build, mvn install, run in prototype mode
• explain dbMock
s
www.devoxx.com

87. www.devoxx.com

88. www.devoxx.com

89. www.devoxx.com

90. Before
m
www.devoxx.com

91. After
m
www.devoxx.com

92. Beyond demoware
For BBC Live Services, the front-end tier (with Kauri) was
actually quite easy
• JaxRS, ≈ 4 resource classes
• Kauri template engine for representation generation
Most of work went in the persistence
• partitioning in Java code (DAO)
• mysqlproxy: master/slave setup
s
www.devoxx.com

93. BBC Live Services
s
www.devoxx.com

94. Kauri Conclusions
Holistic design and collaboration
• from early prototyping to full deployment
• From web-sites to web-services and back (integrated)
Modular construction + wiring
• Choose Routing models for best fit
Standing on the shoulders of
Maven - Spring – JPA – Restlet – jQuery – Blueprint CSS - ...
m
http://www.flickr.com/photos/mushon/282287572/
www.devoxx.com

95. More about Kauri ...
Things we didn't show! (a lot)
• Most notably: forms framework
(client-side RIA framework)
• widget library
• Upcoming; rewriting Daisy on top of Kauri
You're all invited @ http://kauriproject.org/
m
www.devoxx.com

96. x
Concluding statement
Insert concluding statement or quote here.
Remember, this is the final take-away that will
challenge your audience with a call to action.
www.devoxx.com
m

97. We're out to get you!
m
www.devoxx.com

98. Q&A
stevenn@outerthought.org
mpo@outerthought.org
www.kauriproject.org
m+s
www.devoxx.com

99. Thanks for your attention!
www.devoxx.com