Keynote at the European Conference on Cognitive Ergonomics (ECCE 2022), Kaiserslautern, Germany, 7th Oct 2022. https://alandix.com/academic/talks/ECCE2022-keynote/ From prehistory to the internet age, humans have always lived as part of a technologically mediated world. Knapped flints have given way to touch-screens, cuneiform to CSS, but in both rapid hand-eye coordination and long-term social interactions, our experiences and actions in the world are embedded in a physical, mechanical, symbolic and digital nexus. After far too long in the writing, my co-authors and I are delighted that "TouchIT: Understanding Design in a Physical-Digital World" is finally published – symbolic words, recorded in digital media and printed on physical paper. This book covers established and emergent digital technology, but repeatedly the continuity of current and past technology, physical and digital worlds is evident. The fundamental cognitive resources that enable our digital existence in an age of constant flux are the result of aeons of development in a physical world that we remake and reimagine. In this talk I will explore multiple scales of digital interaction from seconds to years, informed by and illuminating what it means to be a fully embodied and richly reflective human.

1. Alan Dix
https://alandix.com/
https://alandix.com/academic/talks/ECCE2022-keynote/
@alanjohndix @CompFoundry
Forever Cyborgs
seeing the world with digital eyes

3. intersecting themes
timescales virtuality
milliseconds
years
digital
mental
embodiment
imagination
memory

5. Fitts’ Law
more than 50 years old!
– Paul Fitts’ paper 1954
– originally physical movement (mouse etc. later)
loads of research in HCI!
– mostly focused on what: numbers, counts
– rarely why – what is going on
what is going on?
– Fitts’ focus information theory – capacity of the brain
(Shannon & Weaver influential at the time)
– alternative cybernetic accounts
– take the body seriously!

6. 1
intended
direction
circle of
error
target
start position
4
finish when
circle of error
within target
actual
direction
2
3
cybernetic movement to target
Note:
• time lag
• stochastic
Fitts’ law if ...
• error prop. to distance
• practiced
• etc. ...
number of steps
~ log ( distance)

7. accurate enough?
think about hands and arms
for mouse initial error 5-20%
initial ballistic movement accurate enough for effectors
– hand for arm, finger-tip for finger
for eye too
saccade accurate enough for effector (fovea)
only need better when artificially extended:
cursor, stylus
Fitts’ is law of cybernetic extension!

8. the extended body

9. the eternal cyborg

11. paths and journeys

13. Alan’s on the road again …
P.S.

14. space and place – absent presence
inhabited paths – traces

16. while walking
active
camera, recorder
passive
GPS, ECG, EDA
when stopped
iPad, blog, flickr,
social media,
uploading data
before/after
web for planning
new contacts on
Twitter, email
reporting, analysis
while walking
other walkers
occasional
friends/family
meeting
when stopped
shop keepers,
B&B proprietors,
chance meetings
talks, visits
before/after
family and friends,
readers,
researchers,
future walkers
social
sphere
technical
sphere

18. http://en.wikipedia.org/wiki/File:Chimp_Brain_in_a_jar.jpg
memory
scattered threads

19. types of memory
seconds
timescale
short-term memory (STM)
long-term memory (LTM)
mezzanine memory
sensory memory
minutes
hours/days
lifetime

20. comparison
short term long term mezzanine
capacity 7+/-2 chunks big! ?
decay ~30s lifetime ? mins–hours
needs rehearsal
laying down implicit repetition implicit
neural electrical physical chemical?
mechanism neuron firing synapse growth LTP

21. ‘normal’ life ‘normal’ life
walking life
universal time
April May June July
2013
more than 3 month gap

22. universal time
April May June July
2013
‘normal’ life ‘normal’ life
walking life
more than 3 month gap
event in past time of
remembering
underestimate
time

23. universal time
April May June July
2013
‘normal’ life ‘normal’ life
walking life
more than 3 month gap

24. ‘normal’ life ‘normal’ life
walking life
more than 3 month gap

25. ‘normal’ life ‘normal’ life
more than 3 month gap
walking life
walking life

26. ‘normal’ life ‘normal’ life
walking life
‘normal’ life

27. many selves ???
• digital extensions into contact lists, email social
media, etc.
allow us to break Dunbar limits
• maybe possible because Dunbar operates per
thread?

29. regret … why?
negative
painful
debilitating

30. Tony Grist, CC0, via Wikimedia Commons
dwelling
on the
past
should
be looking
to the future

31. adaptive?
Gossipguy, CC BY-SA 3.0 via Wikimedia Commons
§
§
§
not necessarily for the
21st century

32. features of regret
• past (bad) decisions brought to mind
• counter-factual “what if” reasoning
• feel bad if could have done better
• feel worse if “just missed”
• most egregious recapitulated

33. tuning learning
to where it
makes the
most difference

34. senses
(3) evaluation
ow! it hurts!
(4) learnt association
touching thorn
is bad
(1) touch thorn (2) thorn pricks
finger
basic reactions - learning
action
emotion

35. NOTE
intention to touch
triggers reaction

36. senses
action
emotion
(3) evaluation
“that hurts”
(1) touch plant
(2) some time
later your finger
is sore
why?
(4) desire to
make sense
regret – the gap

37. senses
(7) learnt association
even though action
not obviously linked
or most salient
(4) logical deduction of
what mattered
determines what is
brought to mind
(6) causes negative
emotion
“if only I hadn’t”
… regret
regret – causal thinking
(5) imagination
causes simultaneous
activation in
relevant areas
action
(6b) counter-factual deduction
of how much it matters
influences strength of emotion
(7b) learnt association
stronger or weaker
depending on
strength of emotion
emotion

38. living in our imagination
what might have been
what might be

40. motivation and
behaviour
back to Rousseau

41. the paradox
tools that
increase motivation
do not always
instil behaviour

42. Jean-Jacques Rousseau
real will
vs.
actual will

43. from real will
to actual will
from motivation
to behaviour
https://commons.wikimedia.org/wiki/File:FitbitStand.jpg
https://commons.wikimedia.org/wiki/File:Duolingo,_mem-referenco,_1.jpeg

44. https://commons.wikimedia.org/
wiki/File:Gamification_Taxonomy.png

45. motivation task orientation
intrinsic extrinsic
locus
of
control
internal
external

46. real will
actual will

48. long-term interaction
more than a moment

49. simple interactions
• limited duration (minutes/hours)
• continuous activity (except breaks/interruptions)
e.g. Norman’s execution evaluation cycle
most design tools and principles
system
evaluation
execution
goal

50. extended interactions
• long duration (days/weeks/years)
• discontinuous activity (multi-phase)
phases of direct interaction
gaps … but things happen

51. examples
• Use of social media
• Shopping sites
• Online courses
• Support applications for physical courses
• ATMs and banks
links to: service design, CRM, adoption and appropriation,
activity theory

52. long term …
• absolute time
– weeks/months/years vs second/minutes
• relative time – discontinuous engagement
– periods of use vs. length of overall activity
– expected/design to be that way (c.f interruptions)

53. time scales
sec min hour day week year
sec
min
hour
day
week
year
absolute length
relative
length
length of overall activity
length of
periods of
engagement
interruption
intermittent

54. long term …
• absolute time
• relative time – discontinuous engagement
• may be multi-scale
– single module vs. university degree
– using city guide in Paris vs. using same app. for tour
of Beijing next year
– activity may be fragmented
– e.g. online course vs support software for physical course

55. Properties (1)
• may cross organizational boundaries
– e.g. hotel and taxi to airport
• dealing with re-engagement
– Triggers
– re-establishing context
• actions outside the digital domain
– e.g. tourist app and visiting site or timetabling app
and attending classes

56. Properties (2)
• learning effects
– but may be occasional use
• activities change – phases/stages
• systems change

58. expert evaluation
start
standard:
cognitive walkthrough
heuristics
between phases maybe finish
overall activity

59. at start
knowledge
– How does the user know the system exists?
motivation
– What makes them want to use it?
opportunity
– What enables actual use? (installation, access etc.)

60. between phases
motivation
– What makes them want to re-engage?
trigger
– What prompts the user to re-engage?
opportunity
– How do they do so? (e.g. find URL, locate app)

61. at end
existence
– Is there a natural end to the overall activity
termination
– If user action is needed is this clear?
recognition
– Does the user know they have got to the end?
satisfaction
– Has the overall activity met the broad goals?

63. Research Questions …

64. synopsis / reminisce
context switching – social media and emotion
Fitts law – the ever extended body
walking – onion rings of experience
memory – threads of experience
regret – living in past and future
motivation – remember Rousseau
long-term interaction – intermittent episodes