Building a "recommended books" application on Google App Engine for Java "early look", using Groovy to help out on the XML processing.

1. Java and Groovy on
Google App Engine
--->
Ken Kousen
ken.kousen@kousenit.com

2. Google App Engine
Google's "cloud computing" solution
Run applications on Google's scalable infrastructure
Pay based on resources used
storage, bandwidth
measured by gigabyte
For free,
up to 500 MB of storage and
up to 5 million page views/month

3. Running Java on GAE
Original GAE version:
Python interpreter and standard libraries
New, "Early Look" version:
JVM available
So Java works, as well as other languages
that compile to the JVM
i.e., Groovy, Scala, JRuby, ...

4. Java "Early Look"
Need to register at http://appengine.google.com
Originally expected 10K developers
Demand was so high, increased to 25K
Now open to ALL
Download SDK
Download Eclipse plugin (optional)
Can run locally or to myapp.appspot.com
If Google Apps user, can map local http address
(example later)

5. Sandbox Environment
Limited access to underlying OS
Manage app through application console
No writing to file system
Can read files uploaded by app
Datastore available (discussed below)
Services available (described next)

6. Welcome to 1999
Let's go back to those thrilling days of yesteryear...
Ricky Martin,
Livin' La Vida Loca
Jar-Jar Binks Bill and Monica

7. Also in 1999
On Dec. 17th, Sun released Servlet 2.2 specification
Established structure of a WAR file
http://www.youtube.com/watch?v=EVESMxs4rbA
Ever since then, you could deploy a WAR file
to an application server
Unfortunately, GAE doesn't know from WAR files

8. WAR, huh, ... what is it good for?
Actually, that's not quite true
GAE does know about war structure,
just not war files
gae_app
src
java code
war
normal war stuff, including classes and lib dirs
WEB-INF
web.xml, etc

9. GAE Limitations
GAE running on Java 6, but with some limitations
Once a request is sent to the client, no further
processing can be done
Request will be terminated if longer than 30 sec to
complete, throwing an exception
Also,
No sockets
No threads or timers
No JNI
System.gc(), System.exit(...), etc, do nothing
Security issues, esp. with class loaders

10. GAE War Quirks
Sample web.xml file
Has DTD based on Web App 2.3
But also includes xmlns and version attributes
(version = 2.5, no less)
It's confused

11. Scalability
About 30 active dynamic requests simultaneously
Average server-side req processing time is 75 ms
--> about 400 requests/sec without additional latency
(note static files not affected by this limit)

12. GAE Services
URL Fetch
URL wrapper that uses Google infrastructure to
retrieve resources
Mail
Send email
Memcache
"high performance in-memory key-value cache"
Image Manipulation
Resize, copy, rotate, flip JPEG and PNG images
User
Detect if user is logged in and is administrator

13. Welcome to the 2300s
Maybe by then we'll have moved beyond relational DBs
(but I doubt it...)

14. Datastore
Distributed data storage
transactional
Filtering and sorting by property value type
NOT a traditional relational database
Google uses the term "schemaless"
Uses optimistic concurrency control with retries

15. GAE and Persistence
GAE doesn't know from relational
All persistence defined by @annotations on classes
JDO
O'Reilly book (c) 2003
JPA
But JDO is the default
(wait, what? JDO? what's up with that?)

16. GAE Persistence
GAE object datastore based on BigTable
BigTable is a massively scalable,
distributed storage system
used by Google for lots of things
For Java, API uses DataNucleus bytecode enhancer
(No, I'd never heard of it either...)

17. Software Development Kit
App Engine SDK
Includes web server (jetty)
Emulates all the GAE services
SDK includes an upload tool to deploy app to GAE
Command line tools included

18. Project Development
Supports Java 5 and Java 6 (Standard Edition)
GAE deployment is Java 6
Google Plugin for Eclipse
version for 3.3 (Europa)
http://dl.google.com/eclipse/plugin/3.3
version for 3.4 (Ganymede)
http://dl.google.com/eclipse/plugin/3.4

19. Project Development
SDK comes as a zip file
Includes sample applications
Some use GWT: Google Web Toolkit
How to do Ajax without doing Ajax
by doing it in Java
GWT also comes with the Eclipse plugin

20. Demo: Recommended Books
Warning: User Interface Poisoning Hazard Ahead

21. Entity Class
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Book {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long id;
@Persistent
private String asin;
@Persistent
private String recommendation;

22. Entity Class
// Other attributes populated by XML response (not persistent!)
private String title;
private String author; // multiple are separated by commas
private String formattedPrice;
private String mediumImageURL;
private String detailPageURL;
// contructors, getter and setter methods
// equals, hashCode, toString overrides as desired
}

23. Entity Class
Book represents book at Amazon.com
Amazon provides Product Advertising API
formerly Amazon Associates Service
http://docs.amazonwebservices.com/AWSECommerceService
RESTful (sort of) web service
Append parameters to request
URL = base?Service=AWSECommerceService
&Operation=ItemLookup
&ASIN=...isbn... // ... etc ...

24. Amazon Web Service
Input URL --> Output XML

25. Amazon Web Service
As of May, 2009
Renamed Product Advertising API (ugh)
As of August, 2009
All REST requests must be digitally signed
Sample code given in Java
Uses javax.crypto classes and Apache Commons Codec

26. DAO Interface
public interface BookDAO {
Book findById(Long id);
Book findByAsin(String asin);
Set<Book> findAllBooks();
Long addBook(Book b);
boolean removeBook(Long id);
}

27. DAO Implementation
public class JdoBookDAO implements BookDAO {
private PersistenceManagerFactory pmf = PMF.get();
@Override
public Long addBook(Book b) {
Book book = findByAsin(b.getAsin());
if (book != null) {
return book.getId();
}
PersistenceManager pm = pmf.getPersistenceManager();

28. DAO Implementation
try {
pm.currentTransaction().begin();
pm.makePersistent(b);
pm.currentTransaction().commit();
} finally {
if (pm.currentTransaction().isActive()) {
pm.currentTransaction().commit();
}
pm.close();
}
return b.getId();
}

29. DAO Implementation
public Set<Book> findAllBooks() {
Set<Book> results = new HashSet<Book>();
PersistenceManager pm = pmf.getPersistenceManager();
Query q = pm.newQuery(Book.class);
q.setOrdering("asin desc");
try {
List<Book> books = (List<Book>) q.execute();
for (Book b : books) {
results.add(b);
}
} finally {
pm.close();
}
return results;
}

30. Use Cases
Servlet implementations of use cases
List all books
Add a new book
Remove a book
Each is mapped in web.xml file
Each goes through a service class to fill in book details
Each forwards to JSP

31. List All Books
public void doGet(
HttpServletRequest req, HttpServletResponse resp)
throws IOException, ServletException {
AmazonBookService service = new AmazonBookService();
Set<Book> books = (Set<Book>) service.getBooks();
req.setAttribute("books", books);
req.getRequestDispatcher("books.jsp").forward(req, resp);
}
What is that AmazonBookService?
It converts XML to Book instances
Enter Groovy...

32. Amazon Book Service
Class implemented in Groovy:
class AmazonBookService {
def baseUrl = 'http://ecs.amazonaws.com/onca/xml'
def params = ['Service':'AWSECommerceService',
'Operation':'ItemLookup',
'AWSAccessKeyId':'... long ugly key ...',
'AssociateTag':'kouitinc-20',
'ResponseGroup':'Medium']
BookDAO dao = DAOFactory.instance.bookDAO

33. Amazon Book Service
def getBooks() {
def books = dao.findAllBooks()
books.each { book ->
book = fillInBookDetails(book)
}
return books
}

34. Amazon Book Service
def fillInBookDetails(Book book) {
def queryString = params.collect { k,v -> "$k=$v" }.join('&')
def url = "${baseUrl}?${queryString}&ItemId=${book.asin}"
def response = new XmlSlurper().parse(url)
def item = response.Items.Item
book.title = item.ItemAttributes.Title
book.author =
item.ItemAttributes.Author.collect { it }.join(', ')
book.formattedPrice =
item.ItemAttributes.ListPrice.FormattedPrice
book.mediumImageURL = item.MediumImage.URL
book.detailPageURL = item.DetailPageURL
return book
}

35. One annoying thing
For Groovy classes,
GAE needs all compiled classes in
war/WEB-INF/classes/ ...
(normal for WAR files)
But couldn't get that and Groovy Eclipse plugin
to keep them in classpath at same time
Had to manually copy compiled classes to proper dir
Grrr... but at least it worked (probably Eclipse issue)

36. JSP
books.jsp displays all the books in the request
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<%@ page isELIgnored="false" %>
...
<c:forEach items="${books}" var="book">
<tr>
<td><a href="${book.detailPageURL}">
<img alt="${book.title} picture"
src="${book.mediumImageURL}">
</a></td>

37. JSP
<td>
<ul>
<li>ISBN: ${book.asin}</li>
<li style="font-weight: bold;">
<a href="${book.detailPageURL}">${book.title}</a></li>
<li style="font-style: italic;">${book.author}</li>
<li>${book.formattedPrice}</li>
</ul>
</td>
<td>${book.recommendation}</td>

38. Resulting Display

39. Admin Capabilities
Use GAE UserService class
UserService us = UserServiceFactory.getUserService();
if (us.isUserLoggedIn()) {
<p><a href="<%=
us.createLogoutURL("/listbooks") %>">Logout</a></p>
} else {
<p><a href="<%=
us.createLoginURL("/listbooks") %>">Login</a></p>
}
Yeah, ugly scriptlet code, but so be it

40. Admin Capabilities
If logged in user and user is registered admin,
Can add new books
Can delete existing books
Otherwise links aren't available
Probably best to make a custom JSP tag out of this
Or use a real security solution
JSecurity, Spring Security, etc.
Easy enough for a demo, though

41. Admin Capabilities
Can set billing limit, too
Mine is $2/day,
so don't bother writing those scripts ... :)

42. Other Services
URL fetch service
Just opening a URL means we're using it already
Has more capabilities for alternate request types
Mail service
Can only send, not receive
Limits on size, frequency, but seem reasonable
Memcache
API implements JCache interface

43. Memcache
Caching capability based on javax.cache package
JCache: JSR 107 (under development)
Cache cache = CacheManager.instance.
cacheFactory.createCache(props)
...
if (cache.get(book.asin)) {
book = cache.get(book.asin)
} else {
book = fillInBookDetails(book)
cache.put(book.asin,book)
}

44. Scheduling
CRON jobs supported in cron.xml
<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
<cron>
<url>/listbooks</url>
<description>Repopulate the cache every day at
5am</description>
<schedule>every day 05:00</schedule>
</cron>
</cronentries>

45. Admin Console
Administrative console located at
http://appengine.google.com for non-Google Apps
or
http://appengine.google.com/a/domain.com
if the domain is using Google Apps
Current limit of 10 apps per user
Can check activity, quotas, etc.

46. GAE vs. Amazon EC2
Amazon EC2 installs images
You need to configure server, JVM, etc.
GAE is a sandbox with provided services
Detailed admin console available
Java persistence framework built in

47. The Quest for the Holy Grails
Grails framework
Spring + Hibernate + Groovy
Modular -- built on plugins
Now there's a GAE plugin
Uninstall hibernate, can then use JDO
Some quirks, but it's early yet
Will eventually be the easiest way to build a GAE app

48. Grails GAE Plugin
1. Create Grails app with matching name
(Workaround available if not possible)
2. Uninstall Hibernate plugin
3. Install app-engine plugin (set APPENGINE_HOME)
4. Set version to 1 (or some other int)
GAE doesn't like decimal version numbers
5. Deploy using package command

49. Opinionated Conclusions: Bad Stuff
Must be able to deploy a WAR file
Hard to take GAE seriously otherwise
Expect this to be easy in release version
Must support the full Java EE standard
Sun (Oracle?) is very upset about this...
Also hard to port existing apps otherwise
Should support other persistence options
Especially Hibernate
Spring would be nice, too (might be asking a lot)
But non-relational issues will persist (sorry)

50. Opinionated Conclusions: Good Stuff
Easy to use cloud framework
Great entry-level tool
Scales like Google
Free!
You get a fully-functioning system for free
Did I mention that it's free?
(okay, within limits, but it's free for a while)

51. Links
Google App Engine home
http://code.google.com/appengine
Source code
git://github.com/kousen/recommended-books.git
Deployed App URL
http://recommended-books.kousenit.com
GAE Plugin for Grails
http://grails.org/plugin/app-engine