In the post-desktop era, many devices (laptops, smartphones, tablet PCs) have proliferated and have become undividable part of our digital lives. However, each device comes with different input methods (multi-touch, mouse, and voice), sensors (accelerometer, compass, GPS) and display sizes. Moreover, they are mostly used for separate tasks, e.g. phones for calling, tablets for reading and web browsing, and laptops for text editing. It may be convenient to use one of the devices for one type of application, while the device may be limited to run the other types of applications. Why shouldn’t we take advantage of all of our personal computing devices and to act as one logical device? In this Virtual Campfire demo, we present a framework for rich internet applications with user interfaces distributed over a federation of heterogeneous commodity devices with multimodality, i.e. laptops, smartphones, and tablet computers. The UI is based on web widgets, running in widget containers such as iGoogle or OpenSocial. We employ the latest Web technologies including XMPP and HTML5 WebSockets to realize cross-platform inter-widget communication, based on the SDK of the EU Project ROLE and i5 Mobile Cloud Infrastructure. This underlying technology virtually connects the distributed UI parts (widgets) and enables real-time input fusion and output fission. We show the framework in action with a prototype for the use case of collaborative semantic video annotation (SeViAnno 2.0), which was already pilot tested for documentation purposes in cultural heritage management. The user has more flexible control over the different parts of the application by using his smartphone or tablet computer with multi-touch functionality, e.g. to navigate over a digital map or preview and annotate videos, and carry out other text input tasks from a laptop.

1. Future Mobile Internet Services
Lehrstuhl Informatik 5
Prof. Dr. M. Jarke
Virtual Campfire
LFG Stadtbaugeschichte
Prof. Dr. M. Jansen
XMPP Inter-widget Communication for Distributed Mobile Web Interfaces
─ Collaborative Semantic Video Annotation Services for Professional Communities
Concepts
One user –
Multiple
personal
computing
devices
Multiple users –
One (logical)
mobile web
application delivered
through the cloud
Federation of heterogeneous personal
computing devices and users’ collaboration
Use Case of Semantic Video Annotation in Cultural Heritage Management
Distribution of UI elements to diverse
(mobile) computing devices
Internet
Real-time remote
collaboration
Tablets for video players
with multi-touch interaction
Collaborative Semantic
Video Annotation
Multimodal interaction with widgets
Personalized and adaptive mobile
web applications
Extension to mobile platforms, e.g.
Android and iOS
Best usage of different mobile devices
Simultaneous real-time interaction with
all UI elements
Mobile multimedia management with
Laptops and PCs for text editing
metadata standards, e.g. MPEG-7
Smartphones for on-site video capture, geotagging on maps and video annotation
Widget-based Technologies
Inter-widget communication (IWC)
Event-based communication between widgets
Widget containers using iGoogle or OpenSocial, etc.
Inter-widget communication (IWC) framework
Local IWC based on HTML5 Web Messaging API
Remote IWC based on XMPP Publish Subscribe XEP
WXG: Gateway for XMPP over HTML5 WebSocket protocol
Library available with EU FP7 ROLE project SDK
Responsive Open
Learning Environments
Rapid development using prevalent web technologies
Ultra High-Speed Mobile Information and Communication