This talk considers the challenges of developing a "canon" for ABM based on research (some of which has been forgotten), the present problem situation of many non comparable models and a possible future based on greater interdisciplinary and more systematic development of methodology.

1. DEPARTMENT OF SOCIOLOGY
The Past, Present and Future of ABM:
How To Cope With A New Research
Method
Edmund Chattoe-Brown (ecb18@le.ac.uk)

2. Plan
• Disagree with me!
• Trying to draw published ideas together:
May be a bit untidy. Sorry.
• Past: How does a “discipline” organise
itself? Does the neoliberal university
“matter” to this?
• Present: A million “toy” models reproducing
the prejudices/division of disciplines.
• Future: Identifying workable “procedures”
for doing ABM.

3. Puzzle 1
Does
anybody
recognise
this?

4. Puzzle 2
What
about this?
Spot the
crucial
difference!

5. The Past
• What would happen if we introduced everyone to
ABM via Hägerstand rather than Schelling? (It is
now on OPENABM at my suggestion.)
• Do you recognise: Clarkson, Gullahorn, Grémy,
Dutton and Starbuck, Bernstein, Loehlin, Kalick and
Hamilton? (And I am not looking hard yet!)
• And (just) now: Hegselmann, R. (2017) ‘Thomas C.
Schelling and James M. Sakoda: The Intellectual,
Technical, and Social History of a Model’, Journal of
Artificial Societies and Social Simulation, 20(3),
<http://jasss.soc.surrey.ac.uk/20/3/15.html>.

6. What Is “Supposed” To Happen?
• “Progress”: People agree what research is important,
how to do “good” work, what “the problems” of the
field are and so on. Over time, it becomes possible to
teach a “canon” (like Marx, Weber and Durkheim) so
a discipline develops a shared sense of identity.
• Too new? How new are we now?
• Too old without success? (Critiques lost too!)
• Too obscure?
• Too pressured to publish?
• Too independent? (Too spread out?)
• What can we do about it now?

7. This Does Matter (At “Level 1”)
• Hägerstrand: Independently calibrated
models really can be validated effectively.
(Still happening, still being ignored: Abdou
and Gilbert.)
• Grémy/Boudon: To justify using ABM look
at the broad pattern of data. Is it explained
more easily by a simple trend?
• Chattoe-Brown: The Zaller-Deffuant model
looks nothing like real data. (I’ll come back
to that.)

8. I Can’t Resist: Abdou and Gilbert (2009)

9. It Also Matters (At “Level 2”)
• ABM can agree its own standards but what
happens if everyone else doesn’t find those
standards credible?
• Do we want to be largely separated from
the rest of social science (like System
Dynamics) or increasingly integrated (like
Social Network Analysis).
• This depends on what we think ABM “is”.
IMO it is a research method. How we do
ABM depends on what we think it is.

10. Connections
• Anecdata: My friend in health HR.
• Laurence Droy: My current PhD student.
• Have people been telling us to “do data” since the
sixties without making an impression? (Does Dutton
and Starbuck report a higher proportion of calibrated
and validated models than Angus and Hassani-
Mahmooei? Uh-oh?) Will the rest of the world get fed
up with us sooner or later? Is later getting sooner?
• Working to survive the over-confidence and rejection
phases. (Is over-confidence more likely now?) AI?
Carley? Helbing? What is the next thing after the Next
Big Thing?

11. What Do We Do?
• This is the easy one: More reading, more citing,
more practical “use” of good examples (in
teaching for example).
• “Recovery” replications: I’m currently doing one
(with Simone Gabbriellini) on Norman
Hummon’s “rational” ABM of social network
formation (2000). Anyone heard of that?
• More attempts to “agree” teaching and
contributions: ESSA sig on education?
• Other: Integrating “non English” ABM/simulation.
• Just being aware of the issue?

12. Quote Maybe About “Empirical” ABM
• “Christianity has not been tried and
found wanting; it has been found
difficult and not tried.” (G. K.
Chesterton)

13. Present
• Opinion dynamics and “element selection”.
• “Rationality” or deliberate decisions: Opinions and
attitudes. Is there a “fact of the matter” involved?
• Media effects (and real events). Feedback loops?
• Membership of groups/parties.
• Psychology: But it doesn’t “agree” (i. e. “backlash”
effects.) Replication crisis?
• (Dynamic) networks.
• Multiple opinions and opinion structure/consistency.
• “Debate”.
• Probably plenty more.
• Can’t just pick some you “like” or ...

14. The Challenge
• Putting x in a model is usually “not implausible”
but leaving it out implies no effect at all (which is
often very implausible).
• Different models for different domains doesn’t
really help. Just “puts the problem back”.
• Models you can calibrate: Apparently no
qualitative data. Is it true, for example, that shift
from “pro” to “anti” traverses “don’t care?”
• What data exists to be explained? (Validation.)
• Methodology: Even validation is better than
nothing.

15. Example: Chattoe-Brown (2014)

16. Let’s Science The Hell Out Of This
• My model is “no good” because I
included elements arbitrarily and
barely calibrated it.
• Ideally my article would not even need
to have been written.
• But my model at least matches
stylised patterns in data (turning
points).
• Please, somebody beat me!

17. Complications
• Lots of legitimate uses of ABM but most of
them are only “intermediates” to empirical
application IMO.
• ABM is very good at formalising theories but a
theory that is complete and coherent still
doesn’t have to be “true” (or “apply”).
• Whatever you say an ABM is for (“interesting
thought experiment”) you have to say what
would count as a success that is more than
personal opinion. (Part of wider “corner cutting”
in academia?)

18. What Do We Do About This?
• Admit it!
• Develop methodology to compare models
(probably has to be empirical).
• At least try to build models that will touch
data (even if you fail). No methodology
without reality. (Survey data example.)
• Connection: If we did calibration and
validation “well” in 1965 (even just once),
what have we been doing since?

19. Future
• ABM tends to take existing research
methods for granted: What do
statisticians (and ethnographers) do
and why do they do that?
• Research design.
• Element selection.
• “Procedural methodology”: This is
what I did and why I did it. Can we
agree it works?

20. Example (Statistics)
• Are English people more reserved than Italians?
• Measure(s) of reserve.
• Pilot survey: What scale of difference do we find
justifying sample size?
• Doing a good (for example unbiased) survey.
• Analysis: Almost the least of it.
• Make sure that the data you need for your analysis
(here just comparison of means) will actually be
produced by your survey.

21. What Is Research Design?
• Lin, Z. and Carley, K. (1995) ‘DYCORP: A Computational
Framework for Examining Organizational Performance
Under Dynamic Conditions’, Journal of Mathematical
Sociology, 20(2-3), pp. 193-217.
• “In an attempt to systematically address what factors affect
organizational performance, we built a dynamic
computational framework for examining organizational
performance in which organizations are composed of
intelligent adaptive agents. Using this framework the user
can contrast organizations with different designs, existing in
different task environments, and subject to different
stresses. We demonstrate the value of this model by
examining how training and stress affect organizational
performance.”
• Am I being unfair? Let’s look more.

22. Methodology: The Next Step
• Many people know the “Gilbert and Troitzsch box” (or
“generative methodology”) but it isn’t so often followed.
• We need to know exactly how this “works” in practice.
• How much can we “fit” models? If we do this don’t we just
end up with a model that matches anything?
• What does sensitivity analysis really tell us?
• What happens if we leave something (media effects) out of
a model? This is OK for calibration and validation (maybe it
works anyway) but for fitting it is “mis-specification”.
• Useful ideas from statistics: Over-fitting, mis-specification,
out-of-sample testing, turning points. How to use these.
• Don’t be downhearted: Generative models may even
predict better (aim to be causal).

23. Example: Switchable models
A model in which we
change only one “process”:
How “dangerous” is leaving
out processes?

24. The Goal
• Methodology will never take the creativity
out of ABM.
• But we need to agree, for example, what
counts as a “match” between real and
simulated data.
• Procedures for converting “personal
opinion” into standards that are hard to
disagree with (but we also need to sort out
exactly what we are disagreeing about
much of which isn’t published.)

25. Example: Bravo et al. (2012)
Real on left: “We built
an experimentLike
model that exactly
replicated the original
experiment with
calibrated parameters.”

26. What Can We Do About This?
• ABM with “research designs”.
• Being as self-critical as possible: Ask yourself why you
assumed something before someone else does.
• Adopting good practice (parameter tables).
• Reality checking: How many papers don’t show real
and simulated data? How many don’t reference
“substantive” research? How many don’t make it clear
how they want to be judged?
• Think about the progressive dimension: Two models
can be “not implausible” separately but not together.
Mark Knopfler: “Two men say they’re Jesus. One of
them must be wrong.”

27. Vision
• ABM competing with each other to improve validation
fit, “strengthen” calibration, test prediction and so on.
• Collaboration between disciplines (based on shared
“process based” approach and methodology) to build
empirically based “modules” for ABM to reduce
“reinventing the wheel”.
• More agreement on what ABM apprentices “need to
know” (and why they need to) not just about the “best”
models but on “how to” as regards ABM building.
• ABM as a specialised but integral part of social science
(and social science in alliance to improve
understanding generally rather than competing to
impose understandings.)

28. Current activities
• Interesting Social Network Analysis in “proper” ABM
(for example changing populations): With SG.
• ABM for torture: Brexit!
• ABM for anti-microbial resistance (funded).
• Integrating models of “place” and social networks (with
Laurence Droy).
• “Switchable” models.
• Opinion dynamics (with Flache, Deffuant, Edmonds et
al.)
• Organisational ecology.
• Target family size: Combining qualitative and
quantitative in ABM.

29. Now Read On 1
• Abdou, M. and Gilbert, N. (2009) ‘Modelling the
Emergence and Dynamics of Social and
Workplace Segregation’, Mind and Society, 8(2),
pp. 173-191.
• Chattoe-Brown, E. (2014) ‘Using Agent Based
Modelling to Integrate Data on Attitude Change’,
Sociological Research Online, 19(1),
<http://www.socresonline.org.uk/19/1/16.html>.
• Chattoe-Brown, E. (2017) 'Agent-Based
Modeling', in Spillman, L. (ed.) Oxford
Bibliographies in Sociology (New York, NY:
Oxford University Press).

30. Now Read On 2
• Chattoe-Brown, E. (in progress) ‘Agent Based
Modelling’. [Currently only from the author.]
• Chattoe-Brown, E. (in progress) ‘Why Questions
Like “Do Networks Matter?” Matter to
Methodology’. [Currently only from the author.]
• Hägerstrand, T. (1965) ‘A Monte Carlo
Approach to Diffusion’, Archives Européennes
de Sociologie, 6(1), pp. 43-67.