Talk presented at the Academic Conference on the Safety of Journalists, World Press Freedom Day, Addis Ababa, 1 May 2019

1. Methodological possibilities for strengthening
the monitoring of SDG indicator 16.10.1
Academic Conference on the Safety of Journalists,
World Press Freedom Day, Addis Ababa, 1 May 2019
Paper authors:
Diana Maynard, Sara Torsner and Jackie Harrison,
Centre for Freedom of the Media (CFOM), The University of Sheffield, UK

2. Background
• The importance of global monitoring of violations against
journalists recognised in SDG agenda (indicator 16.10.1):
‘verified cases of killing, kidnapping, enforced disappearance,
arbitrary detention and torture of journalists, associated media
personnel’
• To meet this ambitious monitoring agenda we need a
comprehensive monitoring system capturing the scope &
nature of violations (lethal & non-lethal)
• Developing such a system confronts us with fundamental
challenges including:
o The accessibility of reliable information on violations
o The systematisation and compiling of information

3. The problem of accessibility
of reliable information
• Substantial gaps in data beyond killings and therefore data on
a range of different types of violations is not readily available
o No single data source systematically captures all categories of 16.10.1
violations (and beyond)
Ø how can this problem be mitigated?
• Practical challenges of collecting and verifying data on
violations:
o Institutionalised local mechanisms for data collection absent or under
development
o Data collected by local civil society actors rarely pooled into a common
repository
o Difficulties with data verification
o Lack of access to datasets
Ø Any attempt to improve monitoring of violations against
journalists in line with 16.10.1 must address the issue of
generating quality data on a range of violations

4. The problem of systematisation
Our research shows:
• Lack of conceptual consistency across datasets
(numerous definitions of instances of violations)
• Substantial difference in the coverage of violation types
across various data sets
• Lack of methodological transparency
• Need for sophisticated data categorisation &
disaggregation to merge data form different sources
o Harmonisation of data categories
o Further categorisation of additional information from
descriptive text sources
• Need to explore and utilize previously untapped data
sources (e.g. by using automated text processing tools)

5. Areas of practical research
1. Events-based methodology
2. Reconciling data from multiple sources
3. Information validity

6. Events-based methodology
• Monitoring violations against journalists traditionally
uses a person-based approach
• This doesn’t preserve relations between individuals
or between events
• Typically used in human rights monitoring (e.g.
HURIDOCS) and in NLP
o What happened (who did what to whom)?
o What actions were taken in response (who did what)?
• An event is a situation in which a specific violation
happens to a specific person, grounded in time
• We can model complex events in a clear way,
preserving temporal and causal relations

7. Events do not happen in isolation
Event 1
Sub-Event 1
Sub-Event 2
Event 1 Event 2
Event 1 Event 2
precedes
causes

8. A complex event
The family of murdered Maltese anti-corruption journalist
Daphne Caruana Galizia is demanding an independent
public inquiry because she had suffered years of
intimidation.
She was killed by a car bomb near her home in October.
Her widely-read blog accused top politicians of corruption.
One of her sons, Paul, said three pet dogs were killed and
attempts were made to burn down the journalist's home.
• Several separate events, but
• are they related?
• was it the same perpetrator(s)?

9. Reconciling information from different sources
Relation Example Issue Solution Same
Event?
Exact match Location: London
Location: London
Identical facts Two events can
be merged
Very
likely
Equivalent
match
Type: murder
Type: assassination
Semantically
equivalent facts
Two events can
be merged
Very
likely
No-conflict Location: London
Date: 2019
Different but
semantically
compatible facts
Facts from two
events can be
merged
Probably
Specificity
Conflict
Location: London
Location: UK
One fact is more
specific than the
other, but both are
semantically
compatible
Facts can be
merged at either
specific or
generic level
Probably
Direct
conflict
Location: London
Location: Paris
Facts conflict
semantically: both
cannot be correct
More info
needed to
resolve conflict
Unlikely

10. Annotation of free text
Name Date Location
City
Location
Country
Event type Type of death
Guerin 26061996 Dublin Ireland Killing Shooting

11. Reconciling databases
Name Date City Country Event type Type of death
Guerin 26061996 Dublin Ireland Killing Shooting
Name Date Organisation Location Event type
Veronica Guerin 26 June 1996 Sunday Independent Ireland Murder
Name Date Organisation Location Event type Sources
Veronica
Guerin
26061996 Sunday
Independent
Dublin,
Ireland
Shooting:
deliberate, fatal
2

12. Information validity
• An information reliability measure is critical for
monitoring
• Can be based on:
o Number of sources that agree
o Trustworthiness of sources (e.g. Global Disinformation Index,
Journalism Trust Initiative)
o Likelihood of error/vagueness (e.g. names are often wrong)

13. Putting it all together

14. Conclusions
• Monitoring the complexity of violations against
journalists by going beyond killings can be achieved
through an events-based approach
• This enables us to investigate patterns, trends and
early warnings, leading to a better understanding of
the contexts in which threats to journalists can
escalate into a killing undertaken with impunity

15. Thank you for listening
Contact:
Diana Maynard (d.maynard@sheffield.ac.uk), Senior Research Fellow,
Centre for Freedom of the Media (CFOM), The University of Sheffield, UK
Sara Torsner (sktorsner1@sheffield.ac.uk), PhD Researcher,
Centre for Freedom of the Media (CFOM), The University of Sheffield, UK
Jackie Harrison (j.harrison@sheffield.ac.uk), Professor Centre for Freedom of the
Media (CFOM), The University of Sheffield, UK
Acknowledgements:
This work has been supported by the European Union’s Horizon 2020 research and
innovation programme under grant agreement No 825297, WeVerify, and by Free
Press Unlimited

16. ICCS Crime Classification Scheme

17. HURIDOCS classification