Presented: April 21, 2016; University of Helsinki at the Northern Data Journalism Conference (NODA16) Academic Pre-Conference This presentation aims at exploring the existing research literature on data journalism. Over the past years this emerging journalistic practice has been established and has also attracted significant attention from journalism scholars. It was time to take a closer look at the existing research literature in order to find out more about how this literature has been developing. Where are the research gaps and what does the future of data journalism research hold? These questions were tackled by carefully selecting a corpus of scholarly literature with empirical foundation in data journalism. This corpus was analyzed with a mixed method approach using qualitative and quantitative techniques. In this way the development of the literature over time could be illustrated and the most influential publications could be identified. Often-used theoretical frameworks and the applied research designs hinted at certain tendencies and gaps in the research literature on data journalism, for example, the dominance of qualitative research design over quantitative ones. Also, a shortcoming of cross-national investigations and ethnographic studies became visible.

1. MEDIA & DESIGN
Research on data journalism:
What is there to investigate?
Insights from a structured literature review
Julian Ausserhofer1,2,3, Robert Gutounig1, Michael Oppermann2,
Sarah Matiasek1,2 & Eva Goldgruber1
1: FH Joanneum University of Applied Sciences, Graz
2: University of Vienna
3: Humboldt Institute for Internet and Society, Berlin
NODA16 Academic Pre-Conference
#NODA16
21.04.2016, University of Helsinki

2. MEDIA & DESIGN
*Tool evaluation partners
*
*
*
Supported by:

3. MEDIA & DESIGN
Research Interest

4. MEDIA & DESIGN
@julauss
Research literature on data journalism
2013: "internalist tendencies at [... the] early
stage of academic research" (Anderson, 2013, p. 1007)
↓
2015: "an explosion in data journalism-
oriented scholarship" (Fink & Anderson, 2015, p. 476)*
"rapidly growing body" of scientific studies
(Lewis, 2015, p. 322)*
*cited via Loosen, Reimer & Schmidt (2015, p. 2)
4#NODA16

5. MEDIA & DESIGN
How is the research literature developing?
What are the research gaps?
Research questions
@julauss 5#NODA16

6. MEDIA & DESIGN
Method

7. MEDIA & DESIGN
Structured literature review
@julauss 7#NODA16

8. MEDIA & DESIGN
"to develop insights, critical reflections,
future research paths and research questions"
(Massaro, Dumay & Guthrie, forthcoming)
It adopts "a replicable, scientific and
transparent process [...] that aims to
minimize bias [...]"
(Tranfield, Denyer & Smart, 2003)
Why a structured literature review?
@julauss 8#NODA16

9. MEDIA & DESIGN
Undertaking a systematic literature review
9
Adapted from Massaro et al. (forthcoming)
Writing a literature review protocol
Developing
insights and
critique
through
analyzing
the dataset
Developing
future
research paths
and questions
Determining
the type of
studies and
carrying out a
comprehensiv
e literature
search
Coding dataDefining the
questions
that the
literature
review
should
answer
@julauss#NODA16

10. MEDIA & DESIGN
10
Developing
insights and
critique
through
analyzing
the dataset
Developing
future
research
paths and
questions
Coding dataDefining the
questions
that the
literature
review
should
answer
Determining
the type of
studies and
carrying out a
comprehensiv
e literature
search
● Empirical research on DDJ
● Social science focus, but open to other
disciplines
● Published after 1995
Included
Journal articles
Book sections
Conference papers
Reports (from industry and
research projects)
PhD theses
Not included
Bachelor's and Master's theses
Press reports
Blog posts
@julauss#NODA16

11. MEDIA & DESIGN
11
Developing
insights and
critique
through
analyzing
the dataset
Developing
future
research
paths and
questions
Coding dataDefining the
questions
that the
literature
review
should
answer
Determining
the type of
studies and
carrying out a
comprehensiv
e literature
search
● Preliminary search with “data-driven
journalism”
● Extracting related terms from the keyword
section of research papers
@julauss#NODA16

12. MEDIA & DESIGN
12
Developing
insights and
critique
through
analyzing
the dataset
Developing
future
research
paths and
questions
Coding dataDefining the
questions
that the
literature
review
should
answer
Determining
the type of
studies and
carrying out a
comprehensiv
e literature
search
Search terms
algorithmic journalism
computational journalism
computer-assisted reporting
data journalism
data-driven journalism
data-driven reporting
database journalism
datajournalism
datenjournalismus
quantitative journalism
No search terms
accountability journalism
crowdsourced journalism
dataviz
datavis
ddj
drone journalism
investigative journalism
online journalism
open journalism
@julauss#NODA16

13. MEDIA & DESIGN
13
Developing
insights and
critique
through
analyzing
the dataset
Developing
future
research
paths and
questions
Coding dataDefining the
questions
that the
literature
review
should
answer
Determining
the type of
studies and
carrying out a
comprehensiv
e literature
search
Scientific Databases
ACM Digital Sowiport
EBSCO Springer
IEEE SpringerLink
JSTOR Taylor & Francis Online
ProQuest Web of Science
Science Direct Wiley
Scopus Google Scholar
Sociological Abstracts
@julauss#NODA16

14. MEDIA & DESIGN
14
Developing
insights and
critique
through
analyzing
the dataset
Developing
future
research
paths and
questions
Coding dataDefining the
questions
that the
literature
review
should
answer
Determining
the type of
studies and
carrying out a
comprehensiv
e literature
search
772 search results
↓
Assessment of title, abstract & keywords
- by two independently working researchers
(Thomas et al., 2004)
↓
33 research publications
@julauss#NODA16

15. MEDIA & DESIGN
15@julauss#NODA16
Records identified
from scientific
databases
(n= 772)
Further publications
from expert poll of
data journalism
researchers (n= 4)
Excluded after
screening
(n= 739)
Preliminary corpus:
publications included
after screening of
records (n= 33)
References from
preliminary corpus
(n = 1151)
Final corpus: Publications
included in the systematic
review (n=40)
Excluded after
screening
(n= 1148)
Further publications
included after screening
of references
(n = 3)
Adapted from Fecher, Friesike & Hebing (2015)

16. MEDIA & DESIGN
16@julauss#NODA16
software-assisted qualitative
content analysis
(Kaefer, Roper, & Sinha, 2015; Mayring, 2000;
Schreier, 2012; QSR International, 2015)
computational analysis of
structural aspects
(Kreibich, 2016; Lopez, 2009)

17. MEDIA & DESIGN
Results

18. MEDIA & DESIGN
Development of the literature over time
n=40
@julauss 18#NODA16

19. MEDIA & DESIGN
Publications by type and citations
n=40
bubble size = number of citations in Google Scholar
@julauss 19#NODA16

20. MEDIA & DESIGN
Affiliations & collaborations
@julauss 20#NODA16

21. MEDIA & DESIGN
1787-2015
n=1644
@julauss
References per year
21#NODA16

22. MEDIA & DESIGN
Most-cited references
Publication
Nr. of
Citations
Meyer, P. (2002/1973). Precision journalism: A reporter’s introduction to social
science methods (4th ed.). Oxford: Rowman & Littlefield.
15
Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good:
“Computer-assisted-reporters” and “programmer-journalists” in Chicago.
New Media & Society, 15(6), 853–871.
15
Gray, J., Bounegru, L., & Chambers, L. (Eds.). (2012). The data journalism
handbook: How journalists can use data to improve the news. Sebastopol:
O’Reilly.
13
@julauss 22#NODA16

23. MEDIA & DESIGN
@julauss
Theoretical frames
● Science and technology studies
● Actor network theory
(Ausserhofer, 2015; De Maeyer, Libert, Domingo, Heinderyckx, & Le Cam, 2015;
Parasie & Dagiral, 2013; Parasie, 2015)
23#NODA16

24. MEDIA & DESIGN
@julauss
Research designs & data collection methods
Method Nr. of studies
In-depth interviews 25
Content analysis 21
Survey 5
Short-term observation 3
Newsroom ethnography 1
Note. Content analysis includes analysis of news, databases, blogs, job ads,
visualizations, briefings, manuals, and more. Short-term observation encompasses visits
to the newsroom and participation in meetings. A newsroom ethnography is defined as a
detailed study of a newsroom over the course of several days.
24#NODA16

25. MEDIA & DESIGN
Geographical scope
Country
Number of
studies
United States 16
United Kingdom 14
Germany 5
International 3
n/a 3
Sweden 2
Switzerland 2
Norway 2
Netherlands 2
… …
@julauss 25#NODA16

26. MEDIA & DESIGN
@julauss 26#NODA16
Research gaps in data journalism research
● Comparision of practices between countries
(Appelgren & Nygren, 2014; Parasie & Dagiral, 2013)
● Long-term studies (Davenport, 2000; Knight, 2015)
● Newsroom ethnographies (Parasie & Dagiral, 2013)
● Software studies (Garrison, 1999; Lewis, 2013; Stavelin, 2013)
● Reader experience studies (Segel & Heer, 2010)

27. MEDIA & DESIGN
Conclusion

28. MEDIA & DESIGN
@julauss 28#NODA16
● data journalism and its investigation has been
developing rapidly
● quality improvements in the research
● issues with the literature: few publications
refer to theory or methodology, just report
what has been investigated
Conclusion

29. MEDIA & DESIGN
@julauss 29#NODA16
● practices in small news organizations, freelancers, local
and mobile data journalism etc.
● gender
● digital methods: investigating the field through its
platforms
● theory
● …
Research opportunities

30. MEDIA & DESIGN
Explore the literature online at:
http://literature.validproject.at
Julian Ausserhofer1,2,3,
julian.ausserhofer@fh-joanneum.at
@julauss
Robert Gutounig1,
@sextus_empirico
Michael Oppermann2,
@oppermann_m
Sarah Matiasek1,2 &
@sarahmatiasek
Eva Goldgruber1
@evagoldgruber
1: FH Joanneum University of Applied Sciences, Graz
2: University of Vienna
3: Humboldt Institute for Internet and Society, Berlin
NODA16 Academic Pre-Conference
#NODA16
21.04.2016, University of Helsinki

31. MEDIA & DESIGN
References
31

32. MEDIA & DESIGN
@julauss 32#NODA16
Anderson, C. W. (2013). Towards a sociology of computational and algorithmic journalism. New Media &
Society, 15(7), 1005–1021. doi: 10.1177/1461444812465137
Appelgren, E., & Nygren, G. (2014). Data journalism in Sweden: Introducing new methods and genres of
journalism into “old” organizations. Digital Journalism, 2(3), 394–405. doi:
10.1080/21670811.2014.884344
Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im
Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale
Methoden in der Kommunikationswissenschaft (pp. 87–111). Berlin: digitalcommunicationresearch.de.
doi: 10.17174/dcr.v2.5
Davenport, L., Fico, F., & Detwiler, M. (2000). Computer–assisted reporting in Michigan daily newspapers:
More than a decade of adoption. Presented at the Association for Education in Journalism and Mass
Communication (AEJMC) National Convention, Phoenix, Arizona.
De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for data journalism:
A qualitative assessment of the anecdotal take-up of data journalism in French-speaking Belgium.
Digital Journalism, 3(3), 432–446. doi: 10.1080/21670811.2014.976415
Fecher, B., Friesike, S., & Hebing, M. (2015). What drives academic data sharing? PLoS ONE, 10(2),
e0118053. doi: 10.1371/journal.pone.0118053
Fink, K., & Anderson, C. W. (2015). Data journalism in the United States: Beyond the “usual suspects.”
Journalism Studies, 16(4), 467–481. doi: 10.1080/1461670X.2014.939852
Garrison, B. (1999). Newspaper size as a factor in use of computer-assisted reporting. Newspaper
Research Journal, 20(3). Retrieved from http://com.miami.edu/car/baltimore1.htm
Kaefer, F., Roper, J., & Sinha, P. (2015). A software-assisted qualitative content analysis of news articles:
example and reflections. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research,
16(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/2123
Knight, M. (2015). Data journalism in the UK: A preliminary analysis of form and content. Journal of Media
Practice, 16(1), 55–72. doi: 10.1080/14682753.2015.1015801
Kreibich, C. (2016). scholar.py. Retrieved from https://github.com/ckreibich/scholar.py

33. MEDIA & DESIGN
@julauss 33#NODA16
Lewis, S. C. (2015). Journalism in an era of big data. Digital Journalism, 3(3), 321–330. doi:
10.1080/21670811.2014.976399
Loosen, W., Reimer, J., & Schmidt, F. (2015). When data become news: A content analysis of data
journalism pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff.
Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for
scholarship publications. In M. Agosti, J. Borbinha, S. Kapidakis, C. Papatheodorou, & G. Tsakonas
(Eds.), Research and Advanced Technology for Digital Libraries (pp. 473–474). Berlin: Springer. Retrieved
from doi: 10.1007/978-3-642-04346-8_62
Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the shoulders of giants: undertaking a
structured literature review in accounting. Accounting, Auditing & Accountability Journal.
Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative
Social Research, 1(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/
view/1089
Parasie, S. (2015). Data-driven Revelation? Epistemological tensions in investigative journalism in the age
of “big data.” Digital Journalism, 3(3), 364–380. doi: 10.1080/21670811.2014.976408
Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted-
reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. doi:
10.1177/1461444812463345
QSR International. (2015). Nvivo. Retrieved from http://www.qsrinternational.com/
Schreier, M. (2012). Qualitative content analysis in practice. Thousand Oaks: SAGE.
Segel, E., & Heer, J. (2010). Narrative visualization: Telling stories with data. IEEE Trans. Visualization and
Computer Graphics, 16(6), 1139–1148. doi: 10.1109/TVCG.2010.179
Stavelin, E. (2013). Computational journalism: When journalism meets programming (Dissertation).
University of Bergen, Bergen.
Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed
management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222.
doi: 10.1111/1467-8551.00375