Keynote Talk at 13th 3IA Conference, July 2010 Design of interactive technologies for a mobile and ubiquitous world

1. Invited talk, Athens 29 May 2010
Design of interactive
technologies for a
mobile and ubiquitous world
Nikolaos Avouris
University of Patras Human-Computer Interaction Group
http://hci.ece.upatras.gr

2. Human-Computer Interaction
Group
University of Patras, Greece

3. Mobile and ubiquitous computing
Ubiquitous computing is a post-desktop model
of human-computer interaction in which
information processing has been integrated into
everyday objects and activities.
– Reliance on context / physical environment
– New kinds of applications (leisure related,
everyday activities)
– Many variations, flavours, technologies
– Focus on devices or specific services
How to design such applications?
3

4. Ubiquitous computing application domains
4

5. Ubiquitous Computing Architecture
Directory services
Digital Information Providers
Spaces
Brokers Directory Service
Providers Provider 1
Service
Provider i
Profile Management
Dynamic Service
User
Service Binding Broker
Profiles
User
Agency
Space Owner System
Physical space
Semantics of Local Service
space Provider j
Space
information Historical user
system interaction data
Physical and sensory data
hyperlink 5

6. Ubiquitous Computing
applications examples
6

7. Museum Game: Inheritance (1/2)
• Collaborative game that requires the players to
discover a special exhibit in a museum based on
hints that must be discovered and collected from
the descriptions of exhibits.
• Direct interaction with physical artifacts- that
become augmented artefacts that are easier to
perceive using the mobile device and physical
hyperlinks
A. Stoica, G. Fiotakis, D. Raptis, I. Papadimitriou, V. Komis, N. Avouris (2007), Field evaluation of
collaborative mobile applications, chapter LVIX in J. Lumsden (ed.), “Handbook of Research on User
Interface Design and Evaluation for Mobile Technology”, pp. 994-1011, Hershey, PA, IGI Global
Papadimitriou I., Komis V., Tselios N., Avouris N., (2006) Designing PDA Mediated Educational
Activities for a Museum Visit, Proceedings of Cognition and Exploratory Learning in Digital Age
(CELDA 2006), December 2006, Barcelona, Spain.(http://www.iadis.org/celda2006/)
7

8. Museum Game: Inheritance (2/2)
8

9. Mobile library assistant (1/2)
• Mobile application to assist library visitors:
– retrieving information about the books by
directly interacting with them
– Searching books similar to those collected
by direct interaction
– Multiple modalities of interaction with the
physical artifacts have been chosen
9

10. Mobile library assistant (2/2)
10

11. Museum Guide (1)
• Investigate multiple modalities for delivering
information (text, images, audio)
• Implement legacy connectors to leverage existing
information.
• Build user profiles
• Physical space navigation support
• Content authoring support
• C. Sintoris, D. Raptis, A. Stoica and N. Avouris, (2007) Delivering Multimedia Content in Enabled
Cultural Spaces,Proceedings of 3rd International Mobile Multimedia Communications Conference,
Mobimedia 2007 , August 27-29, 2007, Nafpaktos, Greece, ACM press.
11

12. Museum Guide (2)
12

13. Game: MuseumScrabble (1)
• A team game involving linking exhibits and
connecting them to topics covered by the
museum
• Deepen the understanding of interaction with
digitally augmented physical artefacts
• Yiannoutsou N., Papadimitriou I., Komis V. and Avouris N., “Playing with” museum exhibits:
designing educational games mediated by mobile technology, Proc. of IDC 2009, 8th International
Conference of Interaction Design and Children, June 3–5, 2009, Como, Italy, pp. 230-233. ACM Press,
New York, NY
• Sintoris C., Stoica A., Papadimitriou I., Yiannoutsou N., Komis V., Avouris N. (2010).
MuseumScrabble: Design of a mobile game for children's interaction with a digitally augmented
cultural space, International Journal of Mobile Human Computer Interaction, 2(2), 53-71, April-June
2010
13

14. MuseumScrabble (2)
14

15. BeNatural (1)
• Personalized mobile shopping assistant.
• Exploration of a new space with different
dynamics.
• Investigation of interaction with a different
physical artefacts (items bought in a
supermarket)
15

16. The beNatural architecture (2)
16

17. Playful Narrative: Who killed Hanae
(Paay et al. 2008)
• Location based narrative. Goal: find the killer
• Navigation in Aalborg city through clue collection
and “suspect interrogation”
• Chapters of a mystery story are “attached” to
different city areas
Street activity Screenshot from the mobile device
17
images from Paay et al., 2008

18. City Game: Frequency 1550
(Huizenga et al. 2009)
• Location based game. Goal: earn citizenship (366
days in Amsterdam)
• Navigation in the historical centre of Amsterdam
• Assignment undertaking in specific areas of the
city
Location activity the Headquarter view during game playing time
18
images from http://freq1550.waag.org/

19. Ubiquitous Computing
design and evaluation
19

20. Abstract design frameworks: An example
Design Principles of Playful Narratives (Yiannoutsou et al. 2010)
Narrative
• Role of user
• Structure of story Playful
character
• Rules of game
• Engagement
Learning
Space • Joy
• thinking
• setting • decision making
• motion Interaction • factual knowledge
building
• with the technology • procedural knowledge
• with the story building
• with the physical space
• social interactions
20

21. Overview of mobile applications design &
evaluation practice ( Avouris et al. 2008)
Source System Number of Evaluation Metrics Used
Participants Method
Jokela et al. Mobile 24 (lab) Laboratory Qualitative measures of user
(2008) Multimedia 15 (field) evaluation and behaviour.
Presentation Field study
Editor
Guo et al. Nintendo 20 Lab based Speed and accuracy in both tasks
(2008) Wiimote and comparative and user preference through
Nunchuk – user study questionnaire
based controller
of a robot
Riegelsberg Use of Google 24 Field study in Qualitative measures and usability
er et al. Maps in Mobile four different problems found using group
(2008) Devices locations. briefing sessions, recorded usage,
multiple telephone interviews and
debriefs in a lab setting
Sanchez et AudioNature, A 10 Case study in Qualitative measures of
al. (2008) pocketPC lab involving effectiveness and performance
device for typical users through pre and post tests and
science learning questionnaires
for the blind
Bellotti et al. Leisure guide 11 Field study Qualitative measures of user
(2008) Magitti over a period experience recorded through
of several questionnaires
days 21

22. Issues to be taken into account
see Framework of (de Sa et al. 2008)
• (1) Locations and Settings: Lighting, Noise, Weather,
Obstacles, Social Environment.
• (2) Movement and Posture: variations for sitting, standing
and walking
• (3) Workloads, Distractions and Activities: Critical
activities, settings or domains requiring different degrees of
attention, Cognitive distractions (e.g., phone ringing, etc),
to study cognitive recovery, Physical distractions
• (4) Devices and Usages: Single vs Dual handed interaction,
and Stylus/Finger/Keyboard/Numeric Pad different mobile
devices (e.g., PDAs, Smart Phones, Portable Media Players,
etc).
• (5) Users and Personas: movement and visual impairment,
Heterogeneity – Age, dexterity, cultural background,
profession etc. 22

23. Adaptation of existing measures
of user performance
• Task Load Index (TLX) extended for mobile
applications, in particular for in car tasks, by addition
of the Distraction scale (Cook 2006)
• Measures related to Field Experimentation method
(Goodman et al. 2004), e.g. Percentage Preferred
Walking Speed (PPWS)
• Measures related to multitasking characteristics
(measures of divided attention)
23

24. New settings for lab
evaluation
(e.g. Kjeldskov &
Stage 2003)
24

25. New requirements for Content Generation:
Create-Attach to Space
25

26. Different evaluation methods and findings
according to the phase of the design process
• early design evaluation
(storyboarding, enactment in
physical setting)
• low fidelity prototype evaluation
(a desktop simulation – no spatial
aspect)
• high fidelity prototype evaluation
(field study) 26

27. Design through enactment
(Jacucci, 2004)
27

28. Storyboard and sketching in design
and evaluation (Buxton, 2007)
28

29. beNatrural Conceptual mobile
application design (1st phase)
29

30. beNatrural Conceptual mobile
application design (2nd phase)
30

31. Low fidelity prototype evaluation
2 dimensional
Mobile
simulation device
of space simulation
31

32. High fidelity prototype evaluation
in the field (Inheritance MuseumGame)
32

33. User 1
User 2
User 3
User n
Field
studies:
Analysis Multimedia
Multimedia
Manager
of user Repository
action
User’s Analyst’s
context System discussions comments
screen
33

34. Dimensions of Analysis
34

35. Use of ActivityLens for data
analysis (Fiotakis et al. 2007)
35

36. Discussion

37. Discussion: On design and evaluation
• .. So far emphasis on user centered interactive
systems design.
• .. typically done by using an evaluation
method to measure or predict how effective,
efficient and/or satisfied people would be
when using the system to perform one or more
tasks.
• Usability evaluation methods range from
laboratory-based user observations, controlled
user studies, and/or inspection techniques
37

38. Some criticism of current practice
• Usability Evaluation can be ineffective and
even harmful if naively done ‘by rule’
rather than ‘by thought’. If done during
early stage design, it can mute creative
ideas that do not conform to current
norms, especially to new fields like
ubiquitous computing
Is usability evaluation considered harmful ? (Greenberg
and Buxton, 2008) .
38

39. Some criticism of current practice
• Current approaches are often assuming
formal task structures more related to
work, while today's’ systems are more
often related to no structured human
activities
• need to move to a more design centered
approach in evaluation (Cockton 2008)
39

40. Some criticism of current practice
• Usability should be related to the value a
product has for its users, as often usable
products are not useful (Cockton 2007, 2008)
• The usability evaluation results should be
judged in terms of their downstream utility
to designers, (Law et al., 2007, Howarth, et
al. 2007)
40

41. Challenges: the way ahead
l Reseting the relation between evaluation and design
l Redefinition of content authoring – interaction
metaphores, evaluation of user experience
l We need to put the activity to the centre of the design
process and not just the user
l There is need for an abstract design method for
ubiquitous computing.
• We need to support the designer and increase the
objectivity with measures of usability, while at the
same time increase the pace of development
41

42. thanks
for more:
hci.ece.upatras.gr 42