Slides from a series of talks for the IET's IoT India Congress and some associated events - SRM Chennai, PES Bengaluru, Srishti Bengaluru. I used different subsets of the slides in each talk - this is the whole deck.

1. Human Machine Inter-Agencies
Exploring the messy areas between people and technology
Dave Murray-Rust, Design Informatics, Edinburgh, 2019

2. What’s in this talk
• Context - a bit about myself
and my group
• Context - Interagencies,
Mess, Research Through
Design
• Social Machines
• Lichtsuchende - Human-robot
interaction
• GeoPact - Smart Contracts in
Action
• Chatty Factories - IoT for
Entangled Ethnography

3. Data driven innovation is transforming society and the
economy. In the Centre for Design Informatics, we design
systems for better human data interaction, in diverse
settings such as health, culture, mobility and finance. We
explore design from, with, and by data: the central concern
is the design of flows of data which sustain and enhance
human values. Relevant technologies range from the
internet of things, through blockchains, to robotics, speech
recognition, data visualisation, interaction design, and social
computing.
“Design from, with, and by data: the
central concern is the design of flows
of data which sustain and enhance
human values.”
Speed, C., & Oberlander, J. (2016). Designing from, with and by Data: Introducing the
ablative framework. In Proceedings of the the 50th Anniversary Conference of the Design

5. Type a quote here.http://blockexchange.designinformatics.org/

6. Larissa Pschetz / Gigbliss - autonomous devices negotiating for energy

7. Bettina Nissen / Trustball - delegating decisions about data sharing

8. Louis Souza / Blockchain Breakfast - manifesting crypto architectures

9. Mark Williams - Quantified Body Language

10. WallVis - modular devices for physicalising data

11. Human-Algorithm Interaction
Exploring the space of socio-technical systems

13. Research Through Design
• Design as a method for generating knowledge
• Based in making and doing
• Can include speculative design, design fiction
–James Auger. 2013. “Speculative design: crafting the speculation”. Digital
Creativity
“speculative designs exist as projections of the
lineage, developed using techniques that focus on
contemporary public understanding and desires,
extrapolated through imagined developments of an
emerging technology”

14. –John Law - After Method: Mess in social science research
“… simple clear descriptions don’t work if what they
are describing is not itself very coherent. The very
attempt to be clear simply increases the mess. …
some things in the world can indeed be made clear
and definite. … But alongside such phenomena the
world is also textured in quite different ways. My
argument is that academic methods of inquiry don’t
really catch these. So what are the textures they are
missing out on?”
Mess

15. Experiential AI
–Hemment, D., Aylett, R., Belle,. V., Murray-Rust, D., Luger, E., Hillston, J.,
Rovatsos, M., Broz, F. (2019). “Experiential AI”. AI Matters. 5: 1. ACM New
York.
“A new research agenda in which artists and
scientists come together to dispel the mystery of
algorithms and make their mechanisms vividly
apparent.”
“The hypothesis is that art can mediate between
computer code and human comprehension to
overcome the limitations of explanations in and for AI
systems”

16. Closed Loop - Jake Elwes

17. Zizi - Jake Elwes (@zizidrag)

18. Zizi - Jake Elwes

19. Thingness
• Things in the world develop and change over time
• Flows of matter, shaped by forces, that shape other things
around them
“… the Old High German word
thing means a gathering, and
specifically a gathering to
deliberate on a matter under
discussion, a contested matter.”
–Martin Heidegger, The Thing, p. 174.

21. Inter-agencies
• Agency - how do things,
people, algorithms affect the
world around them
• Inter-agencies - how do
other things push back?
What are the chains of
effects and reflexive
shapings? What are the
constellations of these
relationships?

22. Speed, C., & Maxwell, D. (2015). Designing through value

23. Navigation - Wayfaring

27. Zhan Guo

28. To say that something is a network is about as
appealing as to say that someone will, from now on,
eat only peas and green beans, or that you are
condemned to reside in airport corridors: great for
traveling, commuting, and connecting, but not to live.
- B. Latour
Networks, Societies, Spheres: Reflections of an Actor-Network Theorist.
International Journal of Communication, 5:796–810, 2011

29. http://carto.metro.free.fr/cartes/metro-london/index.php

31. Example: Hashtags
• Signposts, create areas of joining
• Relate to what has gone before, what comes after
• Bottom up - proposed by the community, informal practices
supported by algorithms

32. Reconfiguration

33. Which side of the API are you
on?
Configuring the
infrastructure?
Programmed by
the
infrastructure?

34. Projects

35. “processes in which the people do the
creative work and the machine does the
administration.[…] The ability to create new
forms of social process would be given to
the world at large, and development would
be rapid.”
Social Machines
– Tim Berners-Lee, Weaving the

36. Social Machines

38. Social Machines
N. Shadbolt, D. Smith, E. Simperl, M. Van Kleek, Y. Yang, and W. Hall, “Towards a classification
framework for social machines,” in SOCM2013: The Theory and Practice of Social Machines,
2013.
Large scale
coordination for
communities

40. “The online world has turned
into an ecosystem of bots”

43. Contextual integrity
online
In the networked
society, you never
know the context
you’re speaking in.
Can algorithms help?
Van Kleek, M., Murray-Rust, D., Guy, A., O'Hara, K., & Shadbolt, N. "Computationally
mediated pro-social deception." ACM CHI2016

44. Deception as a tool for interaction designers
Eytan Adar, Desney S. Tan, and Jaime Teevan. 2013. Benevolent deception in human computer
interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI
'13). ACM, New York, NY, USA, 1863-1872. DOI=http://dx.doi.org/10.1145/2470654.2466246

45. Placebo Buttons?

46. real name & platform policies,
context collapse
Max Van Kleek, Dave Murray Rust, Amy Guy, Daniel A Smith
and Nigel R. Shadbolt. "Self Curation, Social Paritioning",
Escaping from Prejudice and Harassment: the Many
Dimensions of Lying Online", ACM Web Science 2015.

47. experiment - Desire to experiment w/ self representation/identity
I once created a profile under the name and pictures (from the
internet) of a girl, to see how guys talked to girls (p228)
I first created the character to see what it would be like if I was the
complete opposite of who I am. How would people respond, would
the like them or hate their guts? I ended up making quite a few
internet friends and decided to keep living this second life, since
people might get suspicious if I suddenly disappeared. Plus it's still
quite fun to do (p274)
[i joined] as a member of a hate group whom I used as a kind of
psychological experiment in empathy--by performing as a member of
that group, I came to a fuller understanding of what compels their
bigotry. (p461)

48. lieCloud - a (fictional) suite of tools for social deception
lieCal
lieTinerary
liecation
lieMapper
social
steganography
Van Kleek, M., Murray-Rust, D., Guy, A., O'Hara, K., & Shadbolt, N. "Computationally
mediated pro-social deception." ACM CHI2016

50. lieCal
Van Kleek, M., Murray-Rust, D., Guy, A., O'Hara, K., & Shadbolt, N. "Computationally
mediated pro-social deception." ACM CHI2016

51. lieCal
Findings
- design fiction to understand
attitudes and open research
space
- need space for people to
explore identity
- people are very happy to lie to
Facebook
- scared of having a history of lies
- deception by default- create
many deceptive outputs, only tell
the truth when explicitly told
Van Kleek, M., Murray-Rust, D., Guy, A., O'Hara, K., & Shadbolt, N. "Computationally
mediated pro-social deception." ACM CHI2016

52. Acceptability of Digital Phenotyping
“I don’t care if you know about
my battery”
How do we understand the
acceptability of large scale
data collection?
• How do people feel about the technology?
• How burdensome is it?
• Is it ethical?
• Do people understand what the technology
does?
• What do people have to give up to work
with the technology?
• Do they think the technology works? Can
people make use of it?
• Theoretical Framework for Acceptability
Rooksby, John, Alistair Morrison, and Dave Murray-Rust. "Student Perspectives on Digital
Phenotyping: The Acceptability of Using Smartphone Data to Assess Mental Health." CHI 2019.

53. Lichtsuchende -
Human-Robot
Interaction
Murray-Rust, Dave, and Rocio von Jungenfeld. "Thinking through robotic imaginaries." Research
Through Design, 2017.

55. Video at: https://vimeo.com/207512856

58. State Diagram

59. Umwelt
Every subject spins out, like the spider’s threads, its relations
to certain qualities of things and weaves them into a solid
web, which carries its existence
- J. von Uexküll, A foray into the worlds of animals and
humans

62. Findings
- ‘more is different’, emergent
behaviour
- interaction with designers is
important too, balancing
shaping and listening
- playful, open ended
interactions highlight new
kinds of behaviours
- ‘Digital Umwelt’ - how do
electronic devices understand
the world around them?

63. Burnett, D., Thorp, J., Richards, D., Gorkovenko, K. and Murray-Rust, D., “Digital twins as
a resource for design research”. In Proceedings of the 8th ACM International Symposium on

65. Burnett, D., Thorp, J., Richards, D., Gorkovenko, K. and Murray-Rust, D., “Digital twins as a resource for
design research”. In Proceedings of the 8th ACM International Symposium on Pervasive Displays

67. Chatty Speakers
Katya Gorkovenko, Dave
Murray-Rust, Dan Richards,
Dan Burnett

69. As new kinds of
technology increasingly
find their way into the most
intimate, messy, chaotic,
and hard-to-reach parts of
end-users’ daily lives, we
need to ways of
understanding their
effects: how they are
adopted, where they raise
tensions, where
unexpected users emerge,
and so on.

71. Constellations of Production and Use
see: Speed, C. and Maxwell, D. (2015) Designing through value constellations. ACM

72. Entangled Ethnography
(upcoming paper in Ethnographic Praxis in Industry)

73. Findings
- having conversations at
the right time is crucial
(c.f. contextual enquiry)
- things help us to
understand people
- annotating objects with
queries helps think
about what they are
- collecting data around
the object not the
people can be less
invasive

74. Making sense of blockchains

81. Genesis Mining (via Business Insider)

82. Joseph Lindley
- Cryptoheater
(Creativity and
Cognition, 2015)

83. What is important about the
blockchain?
Transactional
Database Multiple
Writers
Absence
of Trust
Disintermediation
Transaction
Interaction
Transaction
Semantics
Choice of
Validators
Asset
Exchange
http://www.multichain.com/blog/2015/11/avoiding-pointless-Blockchain-project/

84. Blockchains are complex
• Cryptographic foundations, social implications
• Different meanings in different communities
• Overlaps with existing technologies
www.pwc.com/us/en/technology-forecast/blockchain/digital-business.html

86. BitBarrista - Larissa Pschetz, Ella Tallyn, Rory Gianni, Chris Speed
Pschetz, Larissa, Ella Tallyn, Rory Gianni, and Chris Speed. "Bitbarista: Exploring perceptions of data
transactions in the internet of things." In Proceedings of the 2017 CHI Conference on Human Factors in
Computing Systems, pp. 2964-2975. ACM, 2017.

87. BitBarrista

88. Tallyn, E., Pschetz, L., Gianni, R., Speed, C., & Elsden, C. (2018). “Exploring Machine Autonomy and
Provenance Data in Coffee Consumption: A Field Study of Bitbarista”. Proceedings of the ACM on Human-
Computer Interaction, 2(CSCW), 170

89. Tallyn, E., Pschetz, L., Gianni, R., Speed, C., & Elsden, C. (2018). “Exploring Machine Autonomy and
Provenance Data in Coffee Consumption: A Field Study of Bitbarista”. Proceedings of the ACM on Human-
Computer Interaction, 2(CSCW), 170

90. Mining and recording transactions
Block Exchange
Understanding
blockchains as a way
to exchange value
- Setup, Exchanges
- Transactions
- Extended Values

91. GeoCoin
Exploring location in the
blockchain
- Warm up
(BlockExchange)
- Guided Experience
- Ideation
- Prototyping
- Credit zones (green) give you money
- Debit zones (red) take money away
- Prizes (black) give money to the first
person who reaches them
Nissen, Bettina, Pschetz, Larissa, Murray-Rust, Dave, Mehrpouya,
Hadi, Oosthuizen, Shaune and Chris Speed. “GeoCoin: Supporting
Ideation and Collaborative Design with Smart Contracts" In Proceedings
of the 2017 CHI Conference on Human Factors in Computing Systems,

92. Findings
- infrastructure matters,
new phones work faster
- blockchain transaction
delays are very visible
- need road safety
briefings for real-world
games
- leave space for stories -
what do the zones
represent?

94. HandFastr - short term marriages on the blockchain.
In collaboration with James Stewart, Max Dovey & Corina Angheloiu.
Video www.vimeo.com/163565402.

95. Findings
- prototyping is really
important
- being ‘just real enough’
- blockchain gives a space
to rethink how things are

96. (free download: https://torquetorque.net/wp-

99. PETRAS Living in the IoT @ Tate Exchange

102. www.pwc.com/us/en/technology-forecast/blockchain/digital-business.html

103. What is a smart contract?
• Digital promises and conditions
• Code that 'does things’ - self executing, defined actions in
response to conditions
• What's special? Security, trust, identity, distribution,
certainty (and money)
• if ( some condition ) then ( some action )
Lots of work here - payments,
change of ownership etc.
Less work here
(outside of the blockchain)

104. Location Based Smart Contracts
• if( the train is a bit late ) then ( partial refund )
• if( the train is very late ) then ( full refund )
• if( 10 active travel journeys ) then ( free bike service )
• if( I leave my car here ) then ( release a bike for 2 hours )
• if( I drive through here ) then ( charge me £2 )

105. Use Cases - active travel
• GeoLockBox: location aware lockable bike basket - if you want to
deliver a parcel near my home, make a contract with my bike
lockbox and pop it in while I'm at work!
• Participatory Infrastructure: the City Council offers a smart
contract - if we can show that several people in the building cycle,
and are prepared to contribute, they will pay half of a bike rack in
front of the building.
• Bike Bus: taking kids to school in a train, with a cyclist at the front
and back - the "drivers" have to be close to all the kids, and get
them to school on time.
• Care Workers move from house to house. They have to phone in
when they get there to prove location. When the system goes
wrong, they have no recourse - need a bottom up system to prove
their location (easily!)

107. System Overview
• Location Verification System
• Location “Predicates” - building
blocks
• Smart Contract Library
• Off-chain server for matching and
connecting

108. Location Predicates
• Key ways to look at location
• A person is in a place
• Any person is in a place - e.g.
race, treasure hunt
• Two people are in the same
place (wherever it is) - privacy
preserving
• Can become true at some point
• Can have constraints about time,
how close, who and what is
involved

109. Smart Contract
Library
• If (…) then (…)
• if( p_colocated( Dave, Courier ) )
then action( open_box )
• Checks predicates against
evidence - verified locations fed
into the contract
• Checking evidence is on chain and
publicly visible
• (but only shows the locations
actually necessary)
• Has Events for all the things you
need to work with predicates

110. Location Server
• Connects between the
blockchain and the rest of the
world (e.g. IoT Infrastructure)
• Listens to Location Based
Smart Contracts - what
predicates are they interested
in?
• Tries to find location updates
from the network that satisfy
the predicates, and pass them
in to the contract
(computationally expensive!)

111. Location Aware Infrastructure
• Mobile devices that use GPS,
Bluetooth Proximity and triangulation
to send location updates
• Network can help verify (e.g.
LoRaWAN can triangulate messages)
• Can send messages back to objects to
do something in response to Smart
Contracts

115. Participatory workshop with GeoPact

117. GeoPact LockBox - Revans, Morgan, Tallyn, Murray-Rust

118. Demo Scenario
• Courier assembling a car - being paid to collect all of the parts for a third
party
• Explore different kinds of location based security

121. Collect the tires
Tires are relatively low value
When the courier is in the right place,
their box opens, and they load the tires in
Then verify using the button that they
have made the collection

122. Collect the chassis
The chassis is higher value - stored in a
locked, automated facility
When the courier is in the right place, the
smart store opens, and they can load in the
chassis
(Then verify using the button that they have
made the collection)

123. Collect the engineThe engine is high value and sensitive - managed
handover
When the courier is in the right place, and so is the
engine supervisor, both boxes open
After the switch, both verify before moving on

131. Object exchanges with GeoPact Boxes

132. GeoPact Scooter Box

134. Possibly the youngest person ever to participate in a smart contract

135. Findings
- working with publics is a crucial
challenge for blockchain systems
- degree of reality is very important
- context is crucial, work with
imaginaries
- what happens when things can act
on their own?
- blockchains give a space to rethink
how things are
-

136. Wrapping Up
• Looked at Human-Algorithm Interaction, Research Through
Design, Experiential AI, using mess, thingness, inter-agency,
constellations
• Social Machines – understanding interaction at web scale
• Lichtsuchende - digital animacy, robot imaginaries, human-robot
interaction
• Chatty Factories - design insights from objects, entangled
ethnography
• GeoPact - prototyping emergin infrastructures, understanding
futures
• Design, making, experience all important to deep understanding
• Looking at inter-agencies gives a rich picture of the world

137. https://www.designinformatics.org/postgraduate/

138. Thanks for listening
Dave Murray-Rust
d.murray-rust@ed.ac.uk
@davemurrayrust
http://dave.murray-rust.org