The paper describes an engineering method for building user interfaces for ubiquitous environments. The method comprises of several extensions in the UsiXML family of modes as well as design and runtime support so as to enable multi-platform, synchronous and collaborative interactions. We demonstrate key concepts of the method and their application by elaborating a scenario of collaborative co-play of the ‘tic-tac-toe’ game. The specific use case features synchronous co-engagement in game play by remote users (players or observers) using desktop PCs or Android devices.

1. Model-based
engineering of
multi-platform, George Vellisa
Dimitrios Kotsalisa
synchronous & Demosthenes Akoumianakisa
Jean Vanderdoncktb
collaborative UIs aDepartment of Applied Informatics & Multimedia,
Technological Education Institution of Crete, Greece
bUniversité catholique de Louvain,
Extending UsiXML for Louvain Interaction Laboratory (www.lilab.eu)
Louvain School of Management, Belgium
polymorphic user interface
specification

2. Introduction
• User interface challenges for software engineers
– Programming intensive
– Broad range of interaction components and widgets
– Platform specificity (GUI objects, Web widgets, etc)
– Lack of interoperability due to platform specificity
• (Some) Model-based engineering claims
– UI engineering = Model specification
– Focus on design rather than implementation
– Linking (rather than direct calls) to platform-specific
libraries

3. Introduction
• User interface challenges for software engineers
– UI development time takes a large portion of time
System
feasibility
Validation
Software plans and
requirements UI Design occupied 45% of the total
Validation
Product design
amount of time devoted to each activity
Verification
Detailed
design
Mean = 44%
Verification
Code
Unit
test
50%
Integration
Product
verification
Implementation
System test
Operations and
maintenance 37%
Revalidation

4. Retrospective assessment
• Claims proven useful
– Model specifications can provide an engineering
basis for UI engineering
– Run-time linking alleviates the complexity of
programming
• Claims proven possible
– Multi-platform execution
– Relatively straightforward Uis (e.g., form-based UI)

5. Retrospective assessment (Cont.)
• Several initiatives
– One of them is UsiXML which is used in this work
• Reference Architectures
– CAMELEON Reference framework www.usixml.eu
www.usixml.org
• Recommended by W3C Working Group on Model-based
User Interface Design
• UsiXML is compliant with this framework
• Plethora of methods with some focus on
collaboration
– FlowiXML, AMENITES, CIAM, TOUCHE

6. Retrospective assessment (cont.)
• Challenges still pending attention
– non-standard and customized widgets?
– synchronous and collaborative UIs?
– new affordances such as social awareness in
distributed collaborative settings?
– radically different interaction vocabularies?
– design and run-time support tools for all the above?

7. Research design
• Theory development
– Build abstraction to capture interaction components as
assemblages of capabilities (or affordances)
– Substantiate such capabilities (or affordances) to
polymporphic instantiation schemes
– Specify polymorphic properties in a model-based fashion
• Engineering framework
– Extend UsiXML as needed to support the new specification
– Provide dedicated design tools and run-time components
– Demonstrate the concept’s validity through pilot scenarios

8. Why abstraction?
• Consider various generations of ‘buttons’
– GUI buttons (in the 90s)
• The (primitive) assemblage of button =
{two-state device to exercise control over functionality}

9. Why abstraction?
• A continuum of abstractions (Cameleon Ref.
Framework)
Task & Task Method triggered: download file
Concepts Classes Object: computer file
Modality-independent
AIO type Control AIO
Abstract User
Interface (AUI) Modality-dependent
Software control AIO Physical control AIO
AIO
Platform-
independent Graphical 2D push button Graphical 3D push button Physical push button
Concrete User CIO type
Interface (CUI) Platform-
HTML pushbutton Windows OSF/Motif VRML97/X3D Software Function
independent
push button XmButton button key key
CIO
Code <input type=submit value=“Download" name=btnG>
Final User
Interface (FUI)
Rendering Download
Down
Download Loa d

10. ‘Adding’ social scent
• Scented button (cf. Willett et al., 2007)
• The (augmented) assemblage of ‘scented’ button =
{ Two-state device to exercise control }
+
{ Collectivity (i.e., who else is in the same state of mind)}

11. ‘Adding’ expression of intention
• ‘Like’ buttons (in the Web 2.0)
• The (new) assemblage of the ‘like’ button =
{Two-state devices to exercise control}
+
{Expression of opinion (i.e. I like this)}

12. Putting them all together
• ‘Like’ buttons with social scent
• The (combined) assemblage =
{Two-state device to exercise control}
+
{Expression of opinion (i.e. I like this)}
+
{Collectivity (i.e., who else is in the same state of mind)}

13. Generalizing the concept
• Why limit our creativity to conventional widgets?
– elastic buttons and their interactive behaviour
Note the
embedded
affordances

14. Adding to complexity
• What happens when an elastic button is to be
instantiated across devices and platforms

15. Motivating question
• How can we …
– model digital assemblages in interaction components?
– instantiate them as the need arises and depending on
context / platform capabilities
– integrate them into specifications as first class elements /
properties
– compile specs that are general enough

16. (One) Answer: Polymorphic
instantiation
• A scheme that relies on implementation agnostic
(i.e., abstract) specifications of UIs
– i.e. an abstract button can be instantiated as
• a conventional GUI button when implemented with the ‘primitive’
assemblage
• a scented button when implemented with the ‘augmented’
assemblage
• a ‘like’ button when implemented with the ‘combined’ assemblage
• At run-time and once user and usage context
parameters are discovered, the implementation
agnostic spec is translated to context-specific
interaction vocabularies using dedicated tools

17. Polymorphic Widget Specifications
Basic concept
• Introduce new widgets and their properties as first-
class design elements
WSL Libraries
Resources
Widget Archive

18. The role of MBUI engineering
• Specify complex capabilities & affordances in a
model-based fashion
– Devise required properties
– Define model elements
– Create specification
– Mapping models

19. The role of MBUI engineering
the case of a socially aware button

20. A new set of models
• Extend UsiXML as needed to support the new
specification
– New set of models
• Widget Resource Model
• Behavior model
– Additional models for collaborative work
• Squad model
• Consistency model
• Abstraction model
• Session model

21. A new design tool

22. A new run-time environment
UI-1 UI-N
Design Environment
UI Design Tool
deployUI
6.attachStates 5.buildUI Widget Spec Tool
Platform 4.getArchives
Runtime Environment
Server Widget Archives Repository P.S
WSL Resources
1.initPS Dependencies
Collaboration Plugin
Collaboration Plugin
UI-Model Repository
Social
Social Abstraction 2.join Social
Social Abstraction
Social Abstraction
Abstraction Notification Queue Social Abstraction
Abstraction
Proxy
Proxy Classes Proxy
Proxy Classes
Proxy Classes
Classes Proxy Classes
Classes
Session Manager
applyChanges Web-Services Layer
distributeChanges Polling Polling
trackChanges Axis2-Framework
Client-1 Server Side Framework (SSF) Client-N

23. UIDL’2011 (simple) desktop scenario

24. MDSD’12 multiplatform scenario (in
the paper)
Player-1:Desktop/PC Player-2: Android/mobile Viewer-1: Desktop/PC

25. Post MDSD’12 scenario
• Collaborative assembly of vacation packages
• User roles
– Business partner (desktop)
– Administrator (desktop or mobile
– Platforms (desktop/swing, mobile/Android)
– Context (synchronous and collaborative)
• Complex and custom widgets (i.e., elastic buttons)
• Synchronous collaboration

26. The desktop case
administrator’s view of vacation package / swing

27. The desktop case
business partners’ view of vacation package / swing

28. The mobile case
administrator’s view of vacation package - Android
Due to space
constraints
certain functions
become
automatically
adapted to vocal
interaction

29. The designer’s role?
Designing for certain affordances
1
2
3
4
5
6

30. The run-time instantiation
the specified affordances enabled
1
2
3 4 5 6
2

31. Future work & acknowledgements
• Current and on-going work
– Advanced toolkits (i.e., visualization)
– Run-time adaptivity and UI plasticity in distributed and
ubiquitous settings
– ‘Social’ games
• Acknowledgements
– The work is supported by ARCHIMEDES III
– Thanks to Université catholique de Louvain (UCL) for
supporting the first two authors’ PhD dissertations
• Due to be submitted in 2013