1. Clouds and Web2.0
II: Case Study and Tutorial
CTS08 Tutorial
Hyatt Regency Irvine California
May 19 2008
Geoffrey Fox, Marlon Pierce
Community Grids Laboratory, School of informatics
Indiana University
http://www.infomall.org/multicore
gcf@indiana.edu, http://www.infomall.org 1
2. Polar Grid: A Web 2.0 Case
Study
Using Microformats, GeoRSS,
REST, and KML
Slides from Yu Ma
3. More Detailed Information
⢠See Marieâs blog:
http://tethealla.blogspot.com/
⢠We use blogs in the lab extensively as lab
books.
4.
5. PolarGrid Datasets
⢠A typical processed dataset consists of:
â One plain text header file describing the overall
system and metadata of the particular chunk of
measurements.
â Multiple JPG images processed from the
corresponding raw data.
â Collection timestamp and fundamental
properties such as waveform and transmitting
(TX) /receiving (RX) antenna IDs are
embedded in file names.
⢠Images are processed by Kansas
Universityâs Matlab-based code, GOAP
6. Approaches
⢠Convert unstructured metadata description
into microformats.
⢠Construct a relational database to enable
fast and dynamic queries.
⢠Provide geographic annotation embedded
into web feeds that can be plotted by
Google Maps and Google Earth
⢠Technologies:
â GeoRSS: RSS syndication with geo-location
â Microformats: extensions to XHTML
7. Microformat Conversion
⢠Microformats capture âsemanticâ metadata
information in XHTML format.
⢠Principals for converted microformats:
â Simple: capture the most basic information unit.
â Reusable: capture shared features among data.
â Composable: when combined, can reflect more
complex relationships among data.
⢠Most of our microformat classes are prefixed
with âpg:â denoting the PolarGrid project
namespace, except when existing generic ones
such as the geo microformat are used.
8. How To Extend XHTML
⢠Use either <div> or <span> tags.
⢠Display of these tags can be controlled
with style sheets
⢠You can also dynamically control these
with JavaScript.
⢠Next generation of browsers will have
built-in support for well-known
microformats such as hCard.
20. Web 2.0 Lessons
⢠Web 2.0 approaches can be applied to
management of scientific information.
⢠With the right choices, you get a lot for
free.
⢠Microformats: simple ways to encode
name-value pairs, build up semantic
descriptions.
⢠GeoRSS: orders data by both time and
space
â Works in all standard RSS/Atom readers.
â Google Maps supports natively
23. Map Key
⢠Red blocks represent browsers and things that run in them
(JavaScript).
â This is the âuserâ level.
â Client side mashups
⢠Green blocks represent Web servers and their applications.
â This is the âdeveloperâ level.
â Server-side mashups.
â These can run on any hosting environment: your web server,
Amazon EC2, Google GAE, etc.
⢠Blue blocks represent third party services.
â This is the âsystem cloudâ layer.
⢠Arrows represent network communications.
â Everything goes over HTTP
â REST, AJAX: communication patterns.
â RSS, ATOM, JSON, SOAP: message format.
25. GData: the API to Google
Services
⢠Google provides an extensive set of online services with well-
known user interfaces.
â Docs
â Spreadsheet
â YouTube
â Blogger/Blogspot
â Calendar
â Picasso Photo Sharing
â GMail
⢠These have programming interfaces as well as user interfaces.
â Server side: Java, .NET, PHP, Python
â Client side: JavaScript
⢠Client interfaces include visual components suitable for mash-
ups
⢠Server-side APIs donât include visual components but do allow
you to make server-side mash-ups.
26. Server-Side Mash-Up Use Case
⢠Imagine you have a Web service that runs a large parallel
finite element code.
â Output can include images, movies, output files, metadata, etc.
â Your web service manages the interaction with the queuing
system (another topicâŚ)
⢠With a server-side mash-up, your Web Service can
â Post status information about your jobs (âStep 521 Completed!â)
on your blog.
â Post URLs pointing to your output files on your blog.
â Upload your visualizations to Picasso and YouTube (and post
links).
â Generate RSS/Atom feeds of the above.
â Post results to Google Calendar
⢠Weâll look at some Java examples.
27. Getting Started with GData
⢠Create a Blog, make sure you can login to YouTube, etc.
⢠You will need a clientID and associated developer key.
â Get these from http://code.google.com/apis/base/signup.html.
⢠Get the Java code
â http://code.google.com/p/gdata-java-client/downloads/list
⢠See any number of Google examples
â http://code.google.com/apis/base/javadevguide.html
⢠One trick: You can get your blogger ID by examining the your Blogâs
Atom feed.
â It will be in the header and look something like this:
<id>tag:blogger.com,1999:blog-19457310</id>
â The blog ID to use in the code below would be 19457310 in this
example.
⢠The full working example is here
â http://communitygrids.blogspot.com/2008/03/googles-gdata-java-
api.html
â We will simplify for pedagogical reasons.
28. Posting Results to Your Blogspot
Blog
String userName="me@gmail.com";
String password= "qwerty"; Replace these with your values
String content=ââŚâ;
GoogleService myService= Login in
new GoogleService("blogger","");
myService.setUserCredentials(userName,password);
Create the entry to insert
Entry myEntry = new Entry();
myEntry.setTitle(new PlainTextConstruct(title));
myEntry.setContent(new PlainTextConstruct(content));
Insert the new entry
using RESTâs POST
URL postUrl = new URL("http://www.blogger.com/feeds/" + blogID +
"/posts/default");
myService.insert(postUrl, myEntry);
29. Getting a YouTube Feed
String feedUrl="http://gdata.youtube.com/feeds/api/videos/";
String
scarFace5sec="http://gdata.youtube.com/feeds/api/videos/egwB7hVIIE
c";
YouTubeService service = new YouTubeService(clientID,
developer_key);
service.setUserCredentials(userName,password);
VideoFeed videoFeed =
youTubeService.getFeed(new URL(feedUrl),VideoFeed.class);
VideoEntry videoEntry = youTubeService.getEntry(new
URL(scarFace5sec),
VideoEntry.class);
//Loop over feed entries and display.
String feedContent=
"Title: " + videoEntry.getTitle().getPlainText()+â";
feedContent+=videoEntry.getMediaGroup().
30. Posting To Google Calendar
CalendarService calService=new CalendarService(clientID);
calService.setUserCredentials(userName,password);
Log in to the calendar service
URL postUrl =
new URL("http://www.google.com/calendar/feeds/"+userName+"/private/full");
CalendarEventEntry myEntry = new CalendarEventEntry();
myEntry.setTitle(new PlainTextConstruct(titleOfEvent));
Create a new Calendar entry
myEntry.setContent(new PlainTextConstruct(contentToPost));
DateTime startTime = new DateTime(new Date());
When eventTimes = new When();
eventTimes.setStartTime(startTime); Create the event time and insert
myEntry.addTime(eventTimes);
CalendarEventEntry insertedEntry = calendarService.insert(postUrl, myEntry);
CalendarEventEntry myEntry2 = new CalendarEventEntry();
String now=(new Date()).toString();
myEntry2.setContent(new PlainTextConstruct("Test post at "+now));
myEntry2.setQuickAdd(true);
An alternative way to post the entry
// Send the request and receive the response:
CalendarEventEntry insertedEntry2 =calendarService.insert(postUrl, myEntry2);
31. Some Other Things You Can Do
with GData Server Code
⢠YouTube
â Search movies by keyword tag, ârelatedâ, categories
â Upload and download movies.
â Put movies into collections
⢠Calendar
â Retrieve events as RSS feed
⢠Spreadsheet
â Remotely retrieve and insert or change row and cell
data.
â Use structured queries to retrieve data ranges.
â Remotely invoke batch operations
32. Making FaceBook
Applications
Using Facebook as a backend
service.
33. Full Examples and More Detailed
Notes
⢠PHP and JavaScript notes
â http://communitygrids.blogspot.com/2008/02/q
uick-facebook-javascript-api-notes.html
⢠JavaScript example:
â http://communitygrids.blogspot.com/2008/02/li
ttle-more-jsfacebook-hacking.html
34. Why Use FaceBook as a
Portal?
⢠Because it has 10âs of millions of users.
â The largest Web 2.0-style Web portal
⢠Easy to create Social Networks: groups of
friends and share applications,
communicate, share information, etc.
â And more importantly, tools for building and
managing these networks
⢠Can leverage many third party applications
â For example, photo albums
â Of course many of them are trivial, silly
35. FaceBook APIs
⢠These come in two flavors
â Embedded: make your application available through
FaceBook.
⢠PHP APIs
â Embedding: use Facebook authentication and social
network data in your application.
⢠JavaScript API
⢠In both cases, you run your application on your
Web server.
â You application do anything you can implement.
â And you can query FaceBook for social network
information, user metadata.
36. Using The JavaScript API
⢠Getting Started:
â You just need a Web server to host your application.
â Register as a developer
â Place Facebookâs xd_receiver.htm on your web server
⢠Make sure the relative path used is correct.
⢠What will happen?
â Your page will run outside of Facebook.
â Anyone loading your application will be directed to Facebook to
login.
â User is then redirected back to your page.
⢠Your application can make calls back to Facebook.
⢠And you can do anything else you want.
â So you could embed this application in a portlet, interact with a
local database,
â You can build your application with .NET or Java Server Faces,
etc.
37. HTML Page Example
<textarea style="width:500px;height:300px;"
id="_traceTextBox">
</textarea>
<script
src="http://static.ak.facebook.com/js/api_lib/F
acebookApi.debug.js" type="text/javascript">
var api = new
FB.ApiClient('<your_key_here>',
'/xd_receiver.htm', null);
âŚ
</script>
38. Interact with FaceBook
api.requireLogin(function(exception) {
api.friends_get(null,getResults);
var myinfo= 'last_name,
first_name,
hometown_location,
work_history,
pic_small';
api.users_getInfo(api.get_session().uid,
myinfo,
getInfo);
});
39. Notes on Previous Slides
⢠The requireLogin, friends_get, and
users_getInfo are FaceBook API calls.
⢠The getResults and getInfo methods are
callback methods that I defined.
â Shown in next slide
⢠The myinfo data structure is taken from
the Facebook XML user profile.
â http://wiki.developers.facebook.com/index.php
/Users.getInfo
40. Developer Defined Call-Back
Functions
function getResults(result,exception) {
Debug.dump(api.get_session().uid,âYou');
Debug.dump(result,âYour Friendsâ);
}
function getInfo(result,exception) {
Debug.dump(result,âYour info');
}
These two methods receive the output returned from
Facebook by the APIâs friends_get() and users_getInfo()
methods. The results are dumped to the text window but
you can do more interesting things.
41. Output Info Dumped to Text
Area
you: 627774031
Your friends: {Array}
[Deleted]
Userâs Facebook ID and the ID numbers of his friends
Your Info: {Array}
[0]: {Object}
first_name: Marlon
hometown_location: {Object}
city:
state:
Userâs profile metadata. Note some fields are strings, some are
country:
Arrays, and some are structured objects of strings.
zip:
last_name: Pierce
pic_small: http://profile.ak.facebook.com/profile6/1797/98/t627774031_2463.jpg
uid: 627774031
work_history: {Object}
42. Facebooksâ Query Language
⢠All of the FaceBook API is basically a set
of wrappers around their SQL like FQL
query language. So you can, if you prefer,
make custom methods out of FQL query
strings.
var myQuery='SELECT name FROM user
WHERE uid='+api.get_session().uid;
api.fql_query(myQuery, getFQLResponse);
getFQLResponse() is a developer-written callback function
43. Working with Facebook Groups
⢠To list information about groups you are a
member of, use the following.
â You can replace the value of uid with any UID
you know.
â 18629081888 is the Group ID for the Open Grid
Forumâs uninteresting Facebook group
api.groups_get(api.get_session().uid,null,getGroup
s);
api.groups_getMembers(18629081888,getGroupM
embers);
44. More Information
⢠The JavaScript documentation is available
from here
â http://wiki.developers.facebook.com/index.php
/JavaScript_Client_Library
⢠It is pretty minimal, but it is essentially the
same as the better documented PHP API.
â Replace PHPâs "." with a "_".
â So PHPâs users.getInfo becomes
users_getInfo(...) in JavaScript.
46. What Is a Gadget?
Simple gadgets for getting a Grid proxy credential and
running remote commands. Both run on my own Web
server.
47. Gadget Definition
<?xml version="1.0" encoding="UTF-8" ?>
<Module>
<ModulePrefs title="MyProxy Gadget Example" scrolling="true"
height="500"/>
<Content type="url"
href="http://156.56.104.143:8080/GTLAB/examples/MyProxyExample.j
sf">
</Content>
</Module>
⢠This XML page is web accessible via
http://156.56.104.143:8080/GTLAB/MyProxyGadget.x
ml
⢠This URL points to the actual application (written in
JSF)
⢠Google uses an HTML IFrame to load the page.
⢠You can also use content type=âhtmlâ for more
integrated applications.
â For example, you can send messages between gadgets in
the client.
48. Gadgets and Open Social
⢠Open Social is a consortium of Facebookâs major
competitors.
â Orkut, MySpace, Hi5, LinkedIn, Friendster, etc.
⢠The Open Social API has two parts
â JavaScript libraries for building social gadgets.
⢠Must run in a gadget container (server)
â REST APIs for exchanging data between container
servers.
⢠The ideas:
â JS APIs let you make portable gadgets that will work
in different containers (Hi5ď LinkedIn) but data will be
different.
â REST will let you export/import data.
49. Getting Started
⢠You need a Gadget container to host your
gadgets.
â LinkedIn is notably selective about the gadgets it will
host.
⢠Orkut provides a sandbox, but you have to upload
your gadgets.
⢠Apache Shindig is an extremely simple Java-
based container.
â De facto reference implementation of Open Social
â Not suitable for production by a long ways.
â But it will work on an air plane.
⢠Shindig is something of a moving target.
â My February notes were obsoleteâŚ
50. Getting Started with Shindig
⢠Visit the Web site for information
â http://incubator.apache.org/shindig
⢠Check out the code with SVN
â mkdir $HOME/shindig; cd shindig
â svn co http://svn.apache.org/repos/asf/incubator/shindig/trunk/
⢠Build and run it with Apache Maven
â Run âmvnâ from $HOME/shindig/
â Run âmvn -Prunâ from $HOME/shindig/java/server
⢠Point browser to
http://localhost:8080/gadgets/files/container/sample1.html and look
through other samples.
â You can also load these HTML files using file:// if you don't want to run
the Jetty server.
⢠Periodically check for updates and rebuild
â svn update
⢠The READMEs have better/more up-to-date information than the
website.
51. What Do You Get?
⢠Shindig actually has two major parts:
â A container that can run social gadgets
⢠This must have access to your user and social
network database.
â A gadget aggregator that displays gadgets
⢠Runs the layout manager.
⢠Think of this as your own iGoogle server.
⢠Access the gadget container
â http://localhost:8080/gadgets/files/samplecont
ainer/samplecontainer.html
52. The gadget is actually served
remotely. This is the module
definition.
The gadget can be
displayed in your
http://localhost:8080
display container.
54. Create Your Layout Manager
var my = {};
Note gadgets are loaded from remote host
containers.
my.gadgetSpecUrls = [
'http://www.google.com/ig/modules/horoscope.xml',
'http://www.google.com/ig/modules/aue07otr.xml',
'http://www.labpixies.com/campaigns/todo/todo.xml'
];
my.LayoutManager = function() {
gadgets.LayoutManager.call(this);
};
my.LayoutManager.inherits(gadgets.LayoutManager);
my.LayoutManager.prototype.getGadgetChrome = function(gadget) {
var chromeId = 'gadget-chrome-' + gadget.id;
return chromeId ? document.getElementById(chromeId) : null;
};
55. Initialize and Render
my.init = function() {
gadgets.container.layoutManager = new
my.LayoutManager();
};
my.renderGadgets = function() {
for (var i = 0; i < my.gadgetSpecUrls.length; ++i) {
var gadget = gadgets.container.createGadget(
{specUrl: my.gadgetSpecUrls[i]});
gadgets.container.addGadget(gadget);
gadgets.container.renderGadget(gadget);
}
};
56. HTML Display
<body onLoad="my.init();my.renderGadgets()">
<h2>Sample: Dynamic Height</h2>
<div id="gadget-chrome-0" class="gadgets-gadget-
chrome"></div>
<div id="gadget-chrome-1" class="gadgets-gadget-
chrome"></div>
<div id="gadget-chrome-2" class="gadgets-gadget-
chrome"></div>
</body>
Shows the 3 Gadgets. As we saw, this will use âflowâ layout and
âchromeâ styling.
57. Hosting Your Own Social
Gadget
⢠Place your widget code here:
â $HOME/shindig/javascript/samplecontainer/exam
ples
⢠After compilation, these will be located here:
â ./target/gadgets/files/samplecontainer/examples.
⢠These will be served up from
â http://localhost:8080/gadgets/files/samplecontain
er/examples/myapp.xml
⢠Myapp.xml is the gadget definition we saw
earlier.
58. Hello World Gadget in Container
Uses flat XML file.
User and his network of
friends provided by the
local container. âHelloâ
chosen based friendâs
preferred
http://localhost:8080/gadgets/files/samplecontainer/samplecontainer.html