The Contextual Wizard of Oz - Using the Wizard of Oz methodology in and with context for User Experience Prototyping

1. The Contextual Wizard of Oz
Doris Zachhuber, Thomas Grill, Ondrej Polacek, Manfred Tscheligi
!
mailto:thomas.grill@sbg.ac.at
12.11.2012
Human Computer Interaction & Usability Unit
http://icts.sbg.ac.at

2. 1
The Wizard of Oz
● - Wizard of Oz - Contextual - Framework
!2 / 23

3. 1
The Wizard of Oz
The Wizard of Oz method ... is a research experiment in
which subjects interact with a (computer) system that
subjects believe to be autonomous, but which is actually
being operated or partially operated by an unseen (hidden)
human being.
• Context:
natural language interfaces
! from futuristic to UbiComp appliances
!
• Problems:
!
Complexity of systems
Two wizard scenarios for complex systems
• Goal
!
for a wizard system:
Bridging the gap between the wizard’s role and the system
implementation
● - Wizard of Oz - Contextual - Framework
!3 / 23

4. 2
Contextual ...
• Taking
context into mind
Study scenarios
! Prototyping
! Simulating context
!
!
➡
... a system that allows to simulate contextual parameters
General statements:
!
• it is done anyway ...
• everybody does it ...
• such systems already exist ...
!
does such a system exist?
● - Wizard of Oz - Contextual - Framework
!4 / 23

5. Table 1: Survey a
Contextual ...
Application specific (*)
Logging
2
Functionality
Wizard
tool
Config
x
x
p
p
x
Study support
x
Graphical
x
x
x
x
Stationary
x
x
?
x
General applicable (G)
HRI (H)
Mathematics (M)
Mobile (Mo)
Multimodal (MM)
LBS (L)
PC application (P)
Speech/Linguistik (S)
Text Analysis (T)
UbiComp (U)
Mobile
x
Analysis
Realtime Control
x
Wizarding
Map Support
x
x
x
x
x
x
Domain
Prototyping
Screen Capture
Name
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Data
Contextual
x
?
x
x
Prototype/System
x
x
p
x
Simulation
Wizard – Auto-Config
Stationary
Multiple Config.
x
?
x
x
x
x
x
x
x
x
x
x
x
x
x
Config
Graphical
x
x
x
x
x
x
x
?
x
x
Wizard
tool
Mobile
A
G
D
G
A
G
D
D
D
A
D
D
A
D
G
S
D
G
Wizarding
DART
iCAP
OpenWizard
ConWIZ
SketchWizard
WebWOZ
CSLU Toolkit
DiaWOz-II
Humaine Woz
Polonius
Suede
Wizard of Oz 2
NEIMO
MuMoWOz
The Mobile Oracle
ActiveStory
Topiary
WOEB
Prototyping
Name
Application specific (*)
Functionality
U,L
U,L
U,MM,Mo
G
Mo
Mo
S
M
T,S
H
S
S
S
U,MM,Mo
G
P
L
Mo,MM
Multiple Config.
Table 1: Survey and Analysis of Wizard of Oz tools
?
?
DART x x x
A
x
x
x
x
x
iCAP x x x
G xx
? x
x
?
x
x
OpenWizard x
Dx x x
x x
x
x
x
x
x
x
x
ConWIZ x x
Gx x x
x
x
x
p
x
x
x
x
x
x
SketchWizard
A
x
x
x
x
x
x
x
x
x
x
x
WebWOZ x
G
x
xx
x
?
?
x
x
x
D
x
x
x
x
x
Legend:
x . . . available, CSLU Toolkitcould not be evaluated
p . . . planned feature, ? . . .
* A . . . developed for one application, D . . . developed for one domain, G . . . generic application
DiaWOz-II
D
?
x
x
x
x
[1]
Thomas Grill, OndrejHumaine Woz
Polacek, Manfred Tscheligi. Conwiz: A tool supporting contextual wizard of oz simulation. Proc. of
D
x
x
x
MUM 2012. ACM, 2012. navigation route, simulated facin the study context (e.g.Polonius
A
x
x
x
tory, smart home, or sits in a car, etc.) The wizard ob23
serves of user interacting - Framework
Suede
x
x
x !5 / x
● - Wizardthe Oz - Contextualwith a mobile WOz Prototype D
CIF Wizard
Scenario - Corridor
Navigation system
Corridor Light
Turn left
Contextual Scenario
Turn left 90º
Main building
Scenario - Main Plaza
Navigation System
Traffic Light
Door Lock
Door
Turn right 90º
Gate
Door
Mobile Wizard
Door
Main entrance
Left
Parking
Notes
Contextual Scenario
Turn right
Forward
Factory
Right
building
observes
Door
Park
Back
Notes
Stop
Gate
Main building
Gate
Go
Main entrance
Door

6. 2
Contextual Wizard of Oz
• Goal:
Usable and configurable system that supports the engineer through
prototyping and evaluation
!
! Generic interface.
No reimplementation of the wizard is needed.
!
! Reduce the workload of the human wizard’s
Support of multiwizard scenarios
!
! Applicable in multiple contexts
• Desktop
• Mobile
• UbiComp
• Field and Lab
!
● - Wizard of Oz - Contextual - Framework
!6 / 23

7. 2
Contextual Wizard of Oz
CIF Wizard
Scenario - Corridor
Navigation system
Corridor Light
Turn left
Contextual Scenario
Turn left 90º
Main building
Forward
Scenario - Main Plaza
Door
Navigation System
Traffic Light
Door Lock
Turn right
Turn right 90º
Gate
Door
Mobile Wizard
Door
observes
Contextual Scenario
Main entrance
Left
Parking
Notes
Factory
Right
building
Park
Back
Notes
Stop
Gate
Go
Gate
Main building
Mobile Wizard
connects to and
communicates
with ConWIZ
server
human wizard
Door
Door
Door
Main entrance
Parking
Contectual
Interaction
Framework
Connector
Factory
building
CIF communicates with
contextual objects
Park
ConWIZ Server controls
WOz prototype!
Gate
WOz Prototype
C F
ConWIZ Server
ConWIZ Server controls
- contextual Objects via CIF
● - Wizard of Oz - Contextual - Framework
!7 / 23

8. 3
Contextual Wizard of Oz
Real World
Control
Module
Wizard Framework
- control the behaviour of
real world objects through
communication with the
interaction platform
integrates &
controls
Interaction
Framework
● - Wizard of Oz - Contextual - Framework
Sim. Control
Module
- start, stop simulation
- set scenes
- set parameters of
objects
controls
Wizard of Oz
Communication
Protocol
Visualization
Framework
!8 / 23

9. 3
Motivation & Goals
• Conduct
user studies in the (simulated) target context
Analyze the interplay of user interaction and selected context
parameters
! Change selected context parameters and adapt interactive systems
(following the ubiquitous computing approach) to evaluate changes
in user behavior
!
!
• Rapid
prototyping
Speed up the development and configuration of interactive systems
and the simulated context in which they are applied.
! Reusing existing software components
!
● - Wizard of Oz - Contextual - Framework
!9 / 23

10. 3
Contextual Interaction Framework
Real World
Control
Module
Wizard Framework
- control the behaviour of
real world objects through
communication with the
interaction platform
Sim. Control
Module
- start, stop simulation
- set scenes
- set parameters of
objects
WOz protocol
Control Module
Control Module
Interaction/
State Analyzer
PlugIns
Interaction
Framework
Visualization
Framework
Feedback
Generator
PlugIns
Scenario
Implementation
defines scenario
Object library
Agent library
Logging
Platform
Scene library
Visualisation
Sensor
1
Scree
nX
Actuat
or X
Sensor
2
Actuat
or X
Sensor
X
...
Audio Sim.
Ext. Interf.
Sim.
Tracking
System X input
Audio
X
● - Wizard of Oz - Contextual - Framework
represented through output devices
i.e. screens, speaker and other actuators
output - feedback
observation/
evaluation
logging may
enhance usability
evaluation and
analysis of the
particular data
!10 / 23

11. 3
Contextual Interaction Framework
Contextual Interaction Framework
Scenario
Configuration /
Study Editor
Context
Simulation
WebServer
FileServer
DB Server
CIF Core
Utilities
Ext. Context
TCP Simulation
(3D App., etc.)
ConWIZ Server
Study Configuration
WOz Protocol
● - Wizard of Oz - Contextual - Framework
Network
Communication
Wizard UI
TCP
WOz Protocol
TCP
Mobile
Wizard
Mobile
Wizard
!11 / 23

12. 3
Contextual Interaction Framework
• CIF
CORE
SOA
! PlugIns, Wiring, Datatypes, ...
! PlugIn Configuration support
!
!
• CIF
Study Editor
Wiring based graphical editor
! Prototyping
! Save and load configurations
! Access to CIF Library (currently in development)
! Study Support
!
!
• CIF
!
Wizard Server (ConWIZ)
CIF Wizard protocol
● - Wizard of Oz - Contextual - Framework
!12 / 23

13. 3
CIF Study Editor
• Wiring
based graphical
editor
• Prototyping
• Save and load
configurations
• Access to CIF Library
(currently in
development)
• Study Support
● - Wizard of Oz - Contextual - Framework
!13 / 23

14. 3
CIF Study Editor
-
+
Library
IFW Study Editor: OG Study Setup
Align Diagram Zoom in
Inspector
Zoom out
Zoom fit
Bundle Library
All
Bundle name
Producers / Sensors
RFID Reader
Distancesensor
Microphone
Activity Meter
Distance Sensor
Active
DB Logger
⚙
string
Status
float
float
float
started
float
int
started
Consumers / Actuators
Converters
Datatype Converter
DB Analyzer
X
Y
Z
...
System bundles
DB Logger
Head Motion Analyser
Camera
⚙
string
raw image
raw image
string
float
int
Wizard
⚙
Wizard Data Converter
string
wzType
wzType
wzType
wzType
wzType
bool
int
Speaker
bool
Wizard Data Converter
Light Sensor
string
float
bool
wzType
float
● - Wizard of Oz - Contextual - Framework
!14 / 23

15. 3
ConWIZ Mobile Wizard
• Configurable
interface
• No adoption of the application
needed for different study
concepts
• Dynamic communication protocol
• Mobile solution
● - Wizard of Oz - Contextual - Framework
!15 / 23

16. 3
Wizard of Oz Mobile App
• Wizard
of Oz – Mobile Client
Control and triggering of contextual events on a mobile device
! Implemented as an app on android tablet
! Communicates with the CIF Wizard Server
!
connect to
wizard server
select
study
select
scene
start
disconnected
study
selection
back
back
exit
end
● - Wizard of Oz - Contextual - Framework
scene
selection
wizard
screen
back
GUI Draft
!16 / 23

17. 3
Wizard of Oz Mobile App
• Check
and modify contextual parameters
Available Wizard
Objects
State of the
object
Modify state
● - Wizard of Oz - Contextual - Framework
!17 / 23

18. 3
Wizard of Oz Mobile App
• Invoke
Prototype functionalities
● - Wizard of Oz - Contextual - Framework
!18 / 23

19. 3
Scenario: factory lab
air lock
table
Operator Guide
measurement
station
RFID
Tags
Wafer
Box
Wafer
Box
RFID
Tags
Trolley
Wizard
RFID
Reader
● - Wizard of Oz - Contextual - Framework
Keyboard
RFID
Reader
Keyboard
MS 2
Light
Wafer
Box
delivery rack
Place
for
Trolley
Keyboard
MS 1
RFID
Tags
implanation
storage
rack
Keyboard
evaluator and wizard space
Flatter
HS 1
RFID
Reader
MS 3
RFID
Reader
!19 / 23

20. 3
Scenario: factory lab
• CIF
in the factory lab
• Human Wizard uses
ConWIZ
Control prototype
(machines)
! Control simulation of
contextual parameters (i.e.
feedback conditions,
vibrations, etc.)
!
!
• Evaluation
of
workflow in a factory
context
! influence of ambient
alerting modes
!
● - Wizard of Oz - Contextual - Framework
[1]
Ewald Strasser, Astrid Weiss, Thomas Grill, Sebastian Osswald,
Manfred Tscheligi. Combining implicit and explicit methods for the
evaluation of an ambient persuasive factory display. In Proc. of AMI
2012, 2012.
!20 / 23

21. 3
Scenario: Mobile
• CIF
running on a Laptop
• Human Wizard uses
ConWIZ
Control prototype
! Control simulation of
environmental parameters
!
!
• Evaluation
of
navigation commands
! usability of mobile wizard
!
[1]
Thomas Grill, Ondrej Polacek, Manfred Tscheligi. Conwiz: A
tool supporting contextual wizard of oz simulation. Proc. of
MUM 2012. ACM, 2012.
● - Wizard of Oz - Contextual - Framework
ConWiz - Mobile Wizard
Evaluation
!21 / 23

22. 3
Contextual Interaction Framework
• Java
based multi-platform enabled framework
• OSGi Equinox
!
• Available
for download as
!
Developer Library
! Standalone version
• Starts with CIF Study editor
! Mobile Android App
!
● - Wizard of Oz - Contextual - Framework
!22 / 23

23. 3
CIF Web
• General
!
Information
Download, Documentation,
Support, ...
!
• Community
!
Forum
!
• CIF
!
Base
PlugIn Library
Place for exchanging
developed functionalities
http://cif.icts.sbg.ac.at/
● - Wizard of Oz - Contextual - Framework
!23 / 23

24. ● Questions?
grazie mille!

25. * Contact
HCI & Usability Unit
ICT&S Center, University of Salzburg
Sigmund-Haffner-Gasse 18
5020 Salzburg, Austria
hci-unit@icts.sbg.ac.at
!
!
!
!
Dr. Thomas Grill
thomas.grill@sbg.ac.at
Human Computer Interaction & Usability Unit
hci-unit@icts.sbg.ac.at

26. ●
This is the end ...
● - Topic 1 - Topic 2 - Topic 3 - Topic 4 - Topic 5
!26 / 23

27. ● Future Work
• More
customizable wizard interface
• Less navigation-based interaction
• More scenario-based interaction
● - Topic 1 - Topic 2 - Topic 3 - Topic 4 - Topic 5
!27 / 23