Keynote at Fifth European Tangible Interaction Studio, ENAC Toulouse. Nov. 7-10 2022. https://alandix.com/academic/talks/ETIS2022-keynote/ For many years interaction design was driven by the abstractions of WIMP (windows, icons, menus, pointer). The details differ on desktop applications, web pages or smart-phones and the ‘pointer’ has evolved from mice to trackpad and touch-based interactions, however, for many digital applications, the central aspects are unchanged. What is different is that the screens we encounter, as Weiser predicted, are everywhere: embedded in physical appliances such as showers and toasters and situated in office walls and building facades. Furthermore, we are often engaging with digital applications that have no obvious screen or where the screen if present is only a small part of the interaction; these include voice assistants, semi-autonomous vehicles, and smart cities. Even where the dominant interaction is focused on a screen, the places where we use them and the physical activities, we are doing fundamentally affect the nature of the interactive experience: using a smartphone while sitting in an armchair and watching television, is very different from thumbing a quick message whilst walking down a busy city road on a rainy night. In this talk I will describe several design techniques and prototype tools that seek to address the physicality of digital interactions including the physical nature of the device itself and the physical context in which it is placed. This will include ‘soft’ formal methods to describe physical aspects of devices, ways to use video to model physical prototypes during early design and tools to encourage designers to keep the context of use in mind even when working on largely screen-based interactions. The talk draws on some long-standing work, parts of the recently published book 'TouchIT: Understanding Design in a Physical-Digital World' (co-authored with Steve Gill, Devina Ramduny-Ellis, and Jo Hare) and the InContext project (in collaboration with Miriam Sturdee and Anna Carter). The latter arose from the realisation that despite the vast number of design tools available, nearly all focus entirely on the screen and wireframes. We are asking "what is the Next Generation of UX design tool?" – perhaps you would like to join this conversation.

1. Alan Dix
https://alandix.com/
https://alandix.com/academic/talks/ETIS2022-keynote/
@alanjohndix
Beyond the Wireframe
tools to design, analyse and prototype physical devices

2. background …

4. topics
• material design
• spatial layout
• the body
• digital artefacts
space and spatial
arrangement
digital artefacts
virtual physicality
material artefacts
and design
the body
physiology

6. physical interactions

7. • material design
• spatial layout
• the body
• digital artefacts
space and spatial
arrangement
digital artefacts
virtual physicality
material artefacts
and design
the body
physiology

8. model physical device states
the device ‘unplugged’
the device ‘unplugged’
switch
UP
DOWN
user pushes
switch up
and down
two visible …
and feelable …
states
physigram

9. model logical system
light
OFF
ON
two states
of the system

10. physical–logical mapping
light
OFF
ON
switch
UP
DOWN
user pushes
switch up
and down

11. exposed state
particularly easy to understand and use
but not always 1–1
light
OFF
ON
switch
UP
DOWN
user pushes
switch up
and down
mapping 1–1

12. controlled state

13. compliant interaction
(1) system state visible through control
(2) system and user have similar effects
press
down
UP
DOWN
press
up
kettle switch
system
down
system
down
BOILING
Temp
< 100
POWER
OFF
POWER
ON
system state

14. 14
bounce-back button
OUT
IN
user pushes
switch in
bounce
back
computer
OFF
ON
(a) (a)
switch
(a)
(z)

15. give it to designers …

16. Covid response
workshops
Anna
Vasilchenko
Adriana
Wilde

17. prototyping physicality
https://alandix.com/labs/physproto/
https://hcibook.net/gen/PhysProto/v0/rockall

18. playing physically virtually

19. behind the scenes
physigram

21. physical context

22. • material design
• spatial layout
• the body
• digital artefacts
space and spatial
arrangement
digital artefacts
virtual physicality
material artefacts
and design
the body
physiology

23. not just the screen
https://commons.wikimedia.org/wiki/File:Cat_stares_at_itself_on_computer_monitor.jpg
https://commons.wikimedia.org/wiki/File:Indoor_location_services_on_mobile_phone_(10928087126).jpg

24. context
matters
https://pixnio.com/media/umbrella-rain-pedestrian-women-walking
https://commons.wikimedia.org/wiki/File:Young_people_in_Hong_Kong_using_smartphones_whilst_walking.png
https://tireetechwave.org/

26. Anna Carter
Alan Dix
Miriam Sturdee

28. in situ
https://www.wales247.co.uk/digital-technology-gives-view-of-future-as-part-of-1bn-swansea-regeneration

31. next generation UX tools
current
state of
the art
new methods and tools
for existing interactions
tools for
new
interactions
physical–digital devices
audio, spatial, intelligent
broader context, tool integration,
connecting to developers

32. InContext
investigate
what is being used?
what next?
build
envisionments
and prototypes
teach
importance of context
in UX deisgn

33. do we need new UX tools?
loads of tools
of lots of different kinds
so what’s the problem?
https://commons.wikimedia.org/wiki/File:Wlm_flows_draft4_2.png

34. UX Design Tools Survey 2021 with 3000+ users
HTTPS://UXTOOLS.CO/SURVEY-2021/
Product Designer
UX Developer
UI Designer

35. Current tool usage
Figma
Sketch
Adobe XD

36. Figma
SKETCH
ADOBE XD

37. beyond the wireframe …
scaffolding new designers
including rich context
enabling different modalities
https://commons.wikimedia.org/wiki/File:Wlm_flows_draft4_2.png

39. investigate
what is being used?
what next?

40. remote workshops

41. CHI Workshop
Photos:
Konstantin Aal
University of Siegen

43. build
envisionments
and prototypes

44. storyboard designs with context

45. tinkering at the edges
https://datatodata.com/builder/

46. ScenarioViewer
screens in context
https://alandix.com/labs/scenario-viewer/
https://hcibook.net/experimental/scenario-viewer/demo/internet-fridge/

47. behind the scenes
transform: matrix3d(0.208836, 0.0683259, 0,
-1.8007e-05, 0.0132405, 0.187365, 0,
1.29374e-05, 0, 0, 1, 0, 840, 175, 0, 1);

48. what next?
workshops => roadmap
senses beyond sight
links to service design
context sketching made easy
early-stage data and architecture
integration … beyond 1970’s filesystem
maybe something for you?
https://commons.wikimedia.org/wiki/File:Bouquet_of_3_Roses.svg

49. Beyond the Wireframe
tools to design, analyse and prototype physical devices
Alan Dix
https://alandix.com/academic/talks/ETIS2022-keynote/