Overview of population approach, with more tech detail of core analytics, done for MSR Cambridge

1. FanBook
60 5
5
30
Zippy
A Population Approach to System Design
Matthew Chalmers

2. FanBook
60 5
5
30
Zippy
A Population Approach to Ubicomp System Design
Matthew Chalmers

3. Plus ça change...
“Users’ control of [ubicomp] systems
should be maximised and, given the
heterogeneity of likely patterns of use, such
systems must also be tailorable by the
user. It is not possible to predict the social
conventions governing the modes of use of
ubiquitous computing that will come into
being, so empirical studies of new
applications [are] essential.”
Perceptions and Expectations of Ubiquitous Computing:
Experiments with BirdDog, a Prototype Person Locator,
O’Shea et al., Proc. IEE ITaP, 1991.

4. Seamful Design: Treasure
Limits, variation and dynamics in wifi infrastructure
made into resources for designers and users

5. Seamful Design: Feeding Yoshi
Limits, variation and dynamics in wifi infrastructure
made into resources for designers and users

6. Seamful Design: Castles
Limits, variation and dynamics in software module
infrastructure made into resources for designers and users

7. Analysis Tools: Replayer

8. Analysis Tools: Replayer

9. Analysis Tools: Replayer

10. Analysis Tools: Replayer

11. Shifting to iPhones

12. Shifting to iPhones

13. Shifting to iPhones

14. Yoshi (2009)

15. 28GB of quantitative log data representing 18,000
hours of use by 182,000 users, and qualitative data
from 8,000 users

16. Analysis tools: SGVis

17. SGVis: Categorising users

18. SGVis: Categorising users

19. SGVis: Categorising users
Calculate centroids of each identified cluster
4 categories of users: top players, commuters,
static players, beginners

20. Seamful Design: FlexBook
Mobile social networking app
Dynamic integration of
software modules at runtime
Shared via our “module store”
server, or ad hoc peer-to-peer
Reviewing and user feedback
Implicit logging of use
Explicit comments and ratings
shown in app and Facebook

21. Post Factory

22. A software engineering revolution
Conventional models of the software development process
all claim that “at some point, a product is delivered to a
user community, at which point, for that revision of the
software, the design process is over.”
Paul Dourish

23. A software engineering revolution
“When we look at an interactive system as an evolving
artifact in use, it follows that the process of design does
not end with the delivery of the system to some
community of users. Instead, it continues as they use and
adapt the system.”

24. In structural terms, what is Firefox?
2011: 85% of users had extended v4 with at least 1 add-on
(or module or plug-in or add-in or component or...)
No single structural definition—or class—defines it, and
no one pattern of use either
Similar situation in IE, Photoshop, Outlook and beyond...

25. Practice ahead of theory and engineering
Established approaches are based on program structure
being the same wherever and whenever it is used
How to deal with the scale of human and software
interaction?
How to reason predictively about structure and use?
How to make tools for designers, evaluators and users
that support adaptation and appropriation?

26. A population approach to system design
Class as a varied population of software structures, being
changed in the real world of use
Class as population:
Dynamic instead of static
Stochastic instead of deterministic and discrete
Coupled with programs in use, instead of standing apart from them
Complementary to (and to be used along with) existing class concepts

27. A simple example: EdinburghFestivalGuide
Modules FestivalEvents & FestivalVenues in 100% of instances
Module NextBusToMyNextVenue in 50% of instances and the
module QuickestWalkingRoute in 40%
All recorded instances have a variable meanLocationOfUse
that shows 90% of its values within the city of Edinburgh

28. The population approach
Multiple representations of a software class (or type)
Tools to use, create or adapt these representations
Social interactions and practices involving those tools

29. The traditional representation
A class is a structure, and all instances conform to it
Data structure: variables or state
Code structure: methods, procedures or functions
These form a structural description of potential behaviour and use
Assumption: exact match between the structure of the class
definition, and the structure of each instance

30. The traditional representation
Tests and changes need only be done on one thing at one time:
the class structure
What’s true for the class is true for the deployed instances
Methodologically convenient but not consistently true

31. Borrowing from biology: population thinking
A species is a varied and changing population of individuals
Assumed to be undergoing continual change and evolvution
Each individual member may differ from others, and variation among
individuals is natural and vital to adaptation
Described by family resemblances among observable attributes
The species can be described by the set of current (or recently
recorded) members: the aggregate of each individual’s genes, size,
colour, shape, etc.

32. A new representation: class as population
A class as a varied and changing population of instances
Assumed to be undergoing continual change and evolvution
Each individual instance may differ from others, and variation among
individuals is natural and vital to adaptation
Described by family resemblances of structure and use
A class can be described by the set of current (or recently recorded)
members: the aggregate of each instance’s structure, context, use, etc.

33. Populations: variation and dynamism
What’s true for one instance may not be for another
90% of unmodified instances crash, but those with module A added
don’t crash. Overall, 75% of instances crash
Analysis results may be different at different times
We advertise A and the percentages; a week later only 50% crash
Results for different contexts may differ
News of module A spreads faster in one country than another, so in
Spain 25% crash while in the US 60% crash

34. New and old representations
Variation/dynamism in the match between class & instances
Probabilistic statements/models, coupled to ongoing use, at
the core of designs, analyses and interactions
Population serves as overarching and connecting concept

35. Populations with family resemblances
Fit with how ‘classes’ are used in everyday language
Moving away from narrow reliance on necessary and
sufficient membership properties
Application to software structures demands a holistic
approach

36. Why ubiquitous computing?
It’s about more than just mobile devices...
Systems that fit with context, interaction and activity
...which are varied and dynamic in ways difficult to predict
The system is part of the user’s context
System interactions, and how users are modelled and presented to
others in them, are resources for new user activity

37. Ubiquitous computing
A holistic view spanning technology, use and users
Mark Weiser: The unit of design should be social people, in their
environment, plus your device
Robin Milner: Here we look for synergy between the societal vision on
the one hand, and the development of scientific models and engineering
principles on the other.

38. New tools and new interactions
Analysis of modules, configurations and contexts
Which configurations and contexts are popular? Which are
problematic? How are they used? Are there clusters that should be
separately managed and designed for?
How does my setup compare to my friends’? Are there configurations
like my own that doesn’t crash so much? How are they used?
Access to modules and usage data
May support free dissemination (P2P) or retain a degree of control
‘Lead users’ keen to try out new or specially instrumented versions

39. A third representation: ostension
One or more population members are pointed out as
examples
Many potential ways to make an ostension
e.g. a category or cluster selected from a population of instances

40. From observed use patterns to software
structure
MapTool is usually run along with DGPSLocationStream
UKTrainSchedule is frequently logged as being used in UK
locations tagged as train station
FestivalEvents and FestivalVenues modules are frequently used
with an unofficial module AlternativeGuideToTheFringe when
the user is in Edinburgh Fringe Festival venues

41. SGVis: Categorising users

42. Girolami: statistical analysis of populations
Inferring significant family resemblances among
disparate features of large dynamic populations
Reducing a chosen set of instances to representative
code and usage data

43. Girolami: statistical analysis of populations
Extending RJMCMC and TI approaches to online
environment via Dirichlet process approach
Heterogeneous data => non-parametric hierarchical
mixture model
Stochastic structures abstracted into formal models
e.g. CTMCs, reliant on MCMC posterior inference

44. Calder: formal modelling of populations
An online framework set within end-user apps
Real-time, critical feedback to users
Longer-term feedback to evaluators and developers

45. Calder: role-based interface automata
Methods are classified as Master, Servant or Valets, and
as Input or Output :
Master (M) actions must be performed
Servant (S) actions are offered
Valet (V) actions are either servant or hidden
Input (?) actions receive data
Output (!) actions send data
45

46. Example: a simple GPS system
(c1,c4).(M,S)loc
0,0,0,0 1,0,0,0
c1.M!display (c1,c8).(M,S)req
(c1,c8).(S,M)map (c8,c5).(M,S)log
3,0,0,0 2,0,2,0 2,0,1,0
46

47. Calder: component replacement
When can we replace/substitute one component by another?
C1 ⊗ C2 ⊗ C3 ⊗... ⊗ Cn ⤳ C1’ ⊗ C2 ⊗ C3 ⊗... ⊗ Cn
C1’ has to offer the environment C2 ⊗ C3 ⊗... ⊗ Cn whatever
the environment needs to work
Formalise work by services specified in a temporal logic
C1 ⊗ Env ⊨ Σ : the current system works
C1’ ⊗ Env ⊨ Σ : replace C1 by C1’ and the system still works
Not aiming to check C1’ ⊗ Env ⊨ Σ, but derive it...

48. Deploy app at a large scale, and collect log data
Infer statistical patterns of module structure and use
Make event-based formal models of module configurations
Make probabilistic judgements about failures/exceptions
Feed patterns and judgements into role-specific tools for
awareness and analysis
Support people in these roles in adapting the app and its use

49. Scenario: FanPhoto
FanPhoto: two core components,
for text and photo sharing
We give it out to 100 people
and they start using it
We start developing new
modules
one for sharing data via Facebook
one for fast compression and sharing of
text, photos and video via MANETs

50. Scenario: FanPhoto
A few participants are
interested in new modules 97
download them and show off 3
that they are trying them out
Analysts keep track via log
data streaming back
start to see a new subcluster
forming

51. Scenario: FanPhoto
▽FanPhoto
▷TextSharing
In the IDE, developers can ▷PhotoSharing
see the FanPhoto class ▷FacebookSharing
Modules used by a minority of ▷MANETmodule
users are shown in grey

52. Scenario: FanPhoto
97
After some time, 3
developers and evaluators
meet to play back several
weeks’ use

53. Scenario: FanPhoto
91
After some time, 9
developers and evaluators
meet to play back several
weeks’ use

54. Scenario: FanPhoto
26 51
After some time, 23
developers and evaluators
meet to play back several
weeks’ use

55. Scenario: FanPhoto
FanBook
60 5
5
After some time,
developers and evaluators 30
meet to play back several
weeks’ use

56. Scenario: FanPhoto
FanBook
60 5
5
After some time,
developers and evaluators 30
meet to play back several
weeks’ use
Zippy

57. Scenario: FanPhoto
A developer sets his
IDE to do automatic
updates
Code defining each FanPhoto
new subclass appears
labelled with users’ and
analysts’ names
FanBook Zippy
linked to tools to analyse
the ostensions that made
them

58. Ontology as structure
“An ontology is a formal specification of a shared
conceptualization” Gruber, via SemanticWeb.org
Traditionally, the core of software design is about
finding the perfect hierarchical structure
One structure for all known uses, contexts and people
Adaptation is the problem of changing it

59. Ontology as process
Population approach couples process and structure
Design process in which structure is a resource for use, and in
which use creates or adapts structure
Giddens’ duality of structure, Heidegger’s hermeneutic circle
Part of the process is adaptation of a structure to new contexts
and uses
A design requirement for ubicomp... and CompSci?

60. Designing for duality of structure
Moving from structure to use is commonplace
That’s what happens when you compile code and users run it
The trick is how to move back again
Making code for a class from a set of instances and usage histories
A process that allows for gradual adaptation of a class
Moving between complementary forms of class definition
Each move is (or should be) a social interaction

61. Code as intension
A class’ code, compiled to create an executable shared among users
Population as extension
The deployed instances, that are adapted by users and which create
log data shared with evaluators and developers (and other users)
Sub-population as ostension
A subset of instances selected by one of these people as
particularly interesting or useful, and shared among them for
comment and use
Reduction to a code signature reflecting family resemblances among
the configurations and logs of that subset

62. Ostension revisited
Most forms of ‘analysis’ afford ostensive definition
A way to focus on a subset of structures, contexts, uses and users
e.g. an evaluator’s video of particular users from a system trial, a
user’s list of friends on Facebook, a developer choosing a name of a
city to customise design for
Log data allow us to link to the software used
Extend previous work that links from video to log data

64. A population approach
A combination of theory, design, evaluation and use
Advances in all four needed for success
A circular process designed for duality of structure
Change and human agency as essential elements of the process
Multiple ways of representing a software class
Tools to use, create or adapt these representations
Users’, evaluators’ and developers’ social interactions and practices
involving those tools... that feed back into low-level representations

65. FanBook
60 5
5
30
Zippy
matthew.chalmers@glasgow.ac.uk
www.dcs.gla.ac.uk/~matthew