SlideShare ist ein Scribd-Unternehmen logo
1 von 57
Downloaden Sie, um offline zu lesen
MEDIA & DESIGN
News Production Workflows in Data-
driven, Algorithmic Journalism
A Systematic Literature Review
Julian Ausserhofer1,2
,
@julauss
Robert Gutounig1
&
@sextus_empirico
Michael Oppermann2
@oppermann_m
1: FH Joanneum University of Applied Sciences Graz
2: University of Vienna
Dubrovnik Media Days, 31.10.15
#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
*Tool evaluation partners
*
*
*
Supported by:
MEDIA & DESIGN
Research Interest
MEDIA & DESIGN
#DuMD2015 & #DUMediaDays @julauss, @sextus_empirico & @oppermann_m 4
"experimental use of algorithms, data and social
science methods" in journalism (Gynnild, 2014, p. 715)
MEDIA & DESIGN
@julauss, @sextus_empirico & @oppermann_m 5
(Cox, 2000)
http://www.computerhistory.org/revolution/early-computer-companies/5/102/445
(Meyer, 1973)
1952
#DuMD2015 & #DUMediaDays
Computational methods in journalism
MEDIA & DESIGN
@julauss, @sextus_empirico & @oppermann_m 6
(Garrison, 1996)
#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
The datafication of journalism
@julauss, @sextus_empirico & @oppermann_m 7
"datafication" of society
(Mayer-Schönberger & Cukier, 2013)
"increased focus on measurement, outcomes
assessment" (Anderson, 2015, p. 363)
"journalism's quantitative turn" (Coddington, 2015)
… has also affected the "infrastructures of
journalism" (Russ-Mohl, 2006, p.199)
#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
Research literature on data journalism
@julauss, @sextus_empirico & @oppermann_m 8
"internalist tendencies at [... the] early stage of
academic research" (Anderson, 2013, p. 1007)
↓
"an explosion in data journalism-oriented
scholarship" (Fink & Anderson, 2015, p. 476)*
"rapidly growing body" of scientific studies
(Lewis, 2015, p. 322)*
*cited via (Loosen, Reimer & Schmidt, 2015, p. 2)
#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
@julauss, @sextus_empirico & @oppermann_m 9
In what way have journalistic workflows and
routines changed due to the broad introduction of
data in the newsroom?
How has this impacted journalistic norms and
ethics, and the skill requirements for journalists?
#DuMD2015 & #DUMediaDays
Research questions
MEDIA & DESIGN
Method
MEDIA & DESIGN
#DuMD2015 & #DUMediaDays @julauss, @sextus_empirico & @oppermann_m 11
Systematic literature review
MEDIA & DESIGN
@julauss, @sextus_empirico & @oppermann_m 12
"to develop insights, critical reflections, future
research paths and research questions"
(Massaro, Dumay & Guthrie, forthcoming)
It adopts "a replicable, scientific and transparent
process [...] that aims to minimize bias [...]"
(Tranfield, Denyer & Smart, 2003)
#DuMD2015 & #DUMediaDays
Why a systematic literature review?
MEDIA & DESIGN
Undertaking a systematic literature review
13
Adapted from Massaro et al. (forthcoming)
Own visualization
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Writing a literature review protocol
Developing
insights and
critique
through
analyzing the
dataset
Developing
future research
paths and
questions
Determining
the type of
studies and
carrying out a
comprehensive
literature
search
Coding dataDefining the
questions that
the literature
review should
answer
MEDIA & DESIGN
14
Developing
insights and
critique
through
analyzing the
dataset
Developing
future research
paths and
questions
Coding data
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Determining
the type of
studies and
carrying out a
comprehensive
literature
search
Defining the
questions that
the literature
review should
answer
In what way have journalistic workflows and
routines changed due to the broad introduction of
data in the newsroom?
How has this impacted journalistic norms and
ethics, and the skill requirements for journalists?
MEDIA & DESIGN
15
Developing
insights and
critique
through
analyzing the
dataset
Developing
future research
paths and
questions
Coding data
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Defining the
questions that
the literature
review should
answer
Determining
the type of
studies and
carrying out a
comprehensive
literature
search
● Empirical research on DDJ
● Social science focus, but open to other disciplines
● Published after 1995
Included
Journal articles
Book sections
Conference papers
Reports (from industry and
research projects)
PhD theses
Not included
Bachelor's and Master's theses
Press reports
Blog posts
MEDIA & DESIGN
16
Developing
insights and
critique
through
analyzing the
dataset
Developing
future research
paths and
questions
Coding data
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Defining the
questions that
the literature
review should
answer
Determining
the type of
studies and
carrying out a
comprehensive
literature
search
● Preliminary search with “data-driven journalism”
● Extracting related terms from the keyword
section of research papers
MEDIA & DESIGN
17
Developing
insights and
critique
through
analyzing the
dataset
Developing
future research
paths and
questions
Coding data
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Defining the
questions that
the literature
review should
answer
Determining
the type of
studies and
carrying out a
comprehensive
literature
search
Search terms
algorithmic journalism
computational journalism
computer-assisted reporting
data journalism
data-driven journalism
data-driven reporting
database journalism
datajournalism
datenjournalismus
quantitative journalism
No search terms
accountability journalism
crowdsourced journalism
dataviz
datavis
ddj
drone journalism
investigative journalism
online journalism
open journalism
MEDIA & DESIGN
18
Developing
insights and
critique
through
analyzing the
dataset
Developing
future research
paths and
questions
Coding data
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Defining the
questions that
the literature
review should
answer
Determining
the type of
studies and
carrying out a
comprehensive
literature
search
Scientific Databases
ACM Digital Sowiport
EBSCO Springer
IEEE SpringerLink
JSTOR Taylor & Francis Online
ProQuest Web of Science
Science Direct Wiley
Scopus Google Scholar
Sociological Abstracts
MEDIA & DESIGN
19
Developing
insights and
critique
through
analyzing the
dataset
Developing
future research
paths and
questions
Coding data
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Defining the
questions that
the literature
review should
answer
Determining
the type of
studies and
carrying out a
comprehensive
literature
search
772 search results
↓
Assessment of title, abstract & keywords
- by two independently working researchers
(Thomas et al., 2004)
↓
33 research publications
MEDIA & DESIGN
20
Determining
the type of
studies and
carrying out a
comprehensive
literature
search
Defining the
questions that
the literature
review should
answer
Developing
future research
paths and
questions
Developing
insights and
critique
through
analyzing the
dataset
Coding data
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
qualitative coding
computational analysis
MEDIA & DESIGN
Results
MEDIA & DESIGN
Literature Corpus
MEDIA & DESIGN
23
Research literature on workflows in DDJ
Development of literature over time
n=33
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
24@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Research literature on workflows in DDJ
Publication type
Parasie &
Dagiral,
2013
Lewis &
Usher, 2013
Diakopolus,
2015
Karlsen &
Stavelin,
2014
n=33
node size = number of citations in Google Scholar
MEDIA & DESIGN
25@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Research literature on workflows in DDJ
Publication year of referenced papers
1787-2015
n=1151
Constitution of the
United States
MEDIA & DESIGN
26@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Research literature on workflows in DDJ
Publication year of referenced papers
1999-2015
n=1004
2013: 176 cited papers
MEDIA & DESIGN
27@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Research literature on workflows in DDJ
Most-cited references
MEDIA & DESIGN
28@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
edges = 165
corpus publications, n=33
references, n = 70
size = nr. of citations from the corpus
Research literature on workflows in DDJ
Corpus and most-cited references
The Journalist as Programmer
Precision Journalism
Accountability through Algorithm
MEDIA & DESIGN
Nr. of studies
Survey 3
In-depth interviews 22
(Particpant) observation 4
Method(s) of data collection
29@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
Geographical scope
30@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
Nr. of studies Nr. of studies
United States 9 Multinational (>4) 2
United Kingdom 9 Norway 1
Germany 4 Netherlands 1
Sweden 2 Argentina 1
Switzerland 2 Austria 1
Finland 2 Belgium 1
MEDIA & DESIGN
Qualitative analysis -
News Production Workflow
MEDIA & DESIGN
News production workflow
Collecting
Collecting digitalized information from
● publicly available data sets
● requested data
● own research or data collected through crowdsourcing
● automated research (scraping)
● leaked data
(17; Loosen, 2015)
32@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
Type of data source
● government or public bureau
● private corporation
● other organizations like NGOs, research institutes
(28, Loosen, 2015)
33@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Collecting
MEDIA & DESIGN
Access to information
in Norway and USA generally considered good (due to freedom of
information legislation)
Barriers to easy data access
● fees
● lacking competencies for exporting data at public offices
● the form data is often delivered (e.g. as PDF)
● lack of legal legal resources to get access to data that is denied
to journalists in the first place
(6; 14)
34@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Collecting
MEDIA & DESIGN
Selecting data
Because data journalists don’t have much time in the newsroom,
they prefer easy-accessible data, e.g. datasets from public bodies.
They are very cautious when it comes to work with data from
companies or private sources.
(14)
35@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Collecting
MEDIA & DESIGN
Methods
● Journalists use statistical programs (e.g. MySQL, Excel, Access,
SPSS, Google Docs, Maps, Google Refine, Google Fusion Tables
etc.)
● Statistical methods and packages including cluster, network &
regression analysis
--> Journalists are quite cautious with it, since they consider this
outside of their field of expertise.
(6, 14)
36@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Analyzing
MEDIA & DESIGN
Results of DDJ research depend in increasing manner on
(graphical) display of (automated) data analysis software
Tools and programming languages
● Developers use Python, Ruby on Rails, JavaScript, HTML
● Used third-party software includes MySQL, Access, Excel,
Caspio, Tableau, Arcview, TimelineSetter etc.
(14, 17, 28)
37@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Building and visualizing
MEDIA & DESIGN
DDJ is connected to a special way of representation which consists
of
● displaying key messages as graphics and/or
● building interactive web applications
Storytelling is done through
(1) interactive web applications with continuous update of data
(2) interactive application based on a stable data set
(3) continuously updated info graphic or
(4) static infographics
(17)
38@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Building and visualizing
MEDIA & DESIGN
● Working alone at small and medium-sized newspapers, and
part of teams at larger news organizations
● New connections between journalists and programmers
● Members of teams usually have complementary backgrounds
(6, 9, 14, 18)
39@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Collaboration
MEDIA & DESIGN
● Rise of "annotative journalism"
● Regarding the collaboration between newsroom and ICT-
department in media corporations data journalists perceive
the relation as troublesome.
Rather data journalists try to bypass ICT department when
installing extra software or setting up databases or servers
(6, 28)
40@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Collaboration
MEDIA & DESIGN
● The goal of DDJ activities is by definition the [very accurate]
journalistic publication
● Publication of data sources and raw data following open data
principles is mentioned too as an integral part of DDJ.
(6, 17, Coddington, 2014)
41@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
News production workflow
Publishing the product
MEDIA & DESIGN
Backgrounds of data journalists
MEDIA & DESIGN
Backgrounds of data journalists
● Data journalists are appx. 5 years younger than the average
age of German journalists
● All of them have started a university program, 80% have also
finished it
● There is no generalizable “data journalism” career path.
Background is mostly either social science or informatics.
● Backgrounds include programming, design, typography, info
graphics, usability, databases, Web and journalism
● USA: dominance of National Institute for Computer-Assisted
Reporting (NICAR)
(14, 17, 18; 6)
43@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
Enabling and Hindering Factors
MEDIA & DESIGN
Enabling and indering factors
Critical factors
● Skills of the involved journalists
● Support networks (internal and external) important for the
development of skillsavailability of data
● Tools
● Legal resources
● Available resources
(6, 14, 17)
45@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
Conclusion
MEDIA & DESIGN
@julauss, @sextus_empirico & @oppermann_m 47#DuMD2015 & #DUMediaDays
● New kind of people arriving in the newsroom
● New skills needed in the newsroom
● Increase of collaborative newswork
● Shift to data-driven approaches (and
epistemologies debates connected to that)
MEDIA & DESIGN
@julauss, @sextus_empirico & @oppermann_m 48
Scope:
● Practices outside of Europe and North America
● The construction of "facts"
● Gender aspects
Methods of data collection:
● Survey
● (Participant) observation
● Content analysis
● Digital methods
#DuMD2015 & #DUMediaDays
Research gaps
MEDIA & DESIGN
49
Explore the (preliminary) corpus
online – it's a protoype:
http://tinyurl.com/valid-survey
http://validproject.at
Julian Ausserhofer1,2
,
@julauss
julian.ausserhofer@fh-joanneum.at
Robert Gutounig1
&
@sextus_empirico
robert.gutounig@fh-joanneum.at
Michael Oppermann2
@oppermann_m
michael.oppermann@univie.ac.at
1: FH Joanneum University of Applied Sciences Graz
2: University of Vienna
Dubrovnik Media Days, 31.10.15
#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
Tools & Literature
50
MEDIA & DESIGN
51
NVIVO [Computer Software]. (2015). Retrieved from http://www.qsrinternational.com/
Bostock, M., Ogievetsky, V., & Heer, J. (2011). D³ data-driven documents.Visualization and Computer
Graphics, IEEE Transactions on, 17(12), 2301-2309.
Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for
scholarship publications. In Research and Advanced Technology for Digital Libraries (pp. 473-474). Berlin:
Springer.
Zotero [Computer Software]. (2015). Retrieved from http://zotero.org/
Tools & software libraries
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
52
Anderson, C. W. (2013). Towards a Sociology of Computational and Algorithmic Journalism. New Media &
Society, 15(7), 1005–1021. http://doi.org/10.1177/1461444812465137
Anderson, C. W. (2015). Between the Unique and the Pattern: Historical Tensions in Our Understanding of
Quantitative Journalism. Digital Journalism, 3(3), 349–363. http://doi.org/10.1080/21670811.2014.976407
Cox, M. (2000). The Development of Computer-assisted Reporting. Presented at the Newspaper Division,
Association for Education in Journalism and Mass Communication, Southeast Colloquium, Chapel Hill, North
Carolina.
Fink, K., & Anderson, C. W. (2015). Data Journalism in the United States: Beyond the "Usual Suspects".
Journalism Studies, 16(4), 467–481. http://doi.org/10.1080/1461670X.2014.939852
Garrison, B. (1996, September). Computer-Assisted Reporting: A National Use Update. Presented at the
National Society of Professional Journalists Meeting, Washington, D.C.
Lewis, S. C. (2015). Journalism In An Era Of Big Data. Digital Journalism, 3(3), 321–330. http://doi.org/10.
1080/21670811.2014.976399
Loosen, W., Reimer, J., & Schmidt, F. (2015). When Data Become News: A Content Analysis of Data Journalism
Pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff.
Mayer-Schönberger, V., & Cukier, K. (2013). Big Data: A Revolution That Will Transform How We Live, Work,
and Think. Boston: Houghton Mifflin Harcourt.
Primary Sources
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
53
Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the Shoulders of Giants: Undertaking a Structured
Literature Review in Accounting. Accounting, Auditing & Accountability Journal.
Meyer, P. (1973). Precision Journalism: A Reporter’s Introduction to Social Science Methods (2nd ed.). Oxford:
Rowman & Littlefield.
Russ-Mohl, S. (2006). The Economics of Journalism and the Challenge to Improve Journalism Quality: A
Research Manifesto. Studies in Communication Sciences, 6(2), 189–208.
Thomas, J., Harden, A., Oakley, A., Oliver, S., Sutcliffe, K., Rees, R., … Kavanagh, J. (2004). Integrating
Qualitative Research with Trials in Systematic Reviews. BMJ : British Medical Journal, 328(7446), 1010–1012.
http://doi.org/10.1136/bmj.328.7446.1010
Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a Methodology for Developing Evidence-Informed
Management Knowledge by Means of Systematic Review. British Journal of Management, 14(3), 207–222.
http://doi.org/10.1111/1467-8551.00375
Primary Sources
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
54
(34) Aitamurto, T., Sirkkunen, E., & Lehtonen, P. (2011). Trends in Data Journalism. Retrieved from http://virtual.vtt.fi/virtual/nextmedia/Deliverables-2011/D3.
2.1.2.B_Hyperlocal_Trends_In%20Data_Journalism.pdf
(2) Ananny, M., & Crawford, K. (2015). A Liminal Press: Situating news app designers within a field of networked news production. Digital Journalism, 3(2),
192–208. http://doi.org/10.1080/21670811.2014.922322
(12) Appelgren, E., & Nygren, G. (2012). Data Journalism in Sweden – Opportunities and Challenges: A Case Study of Brottspejl at Sveriges Television (SVT). In C.
Mills, M. Pidd, & E. Ward (Eds.), Proceedings of the Digital Humanities Congress 2012. Sheffield: HRI Online Publications. Retrieved from http://www.
hrionline.ac.uk/openbook/chapter/dhc2012-appelgren
(11) Appelgren, E., & Nygren, G. (2014). Data Journalism in Sweden: Introducing New Methods and Genres of Journalism into “Old” Organizations. Digital
Journalism, 2(3), 394–405. http://doi.org/10.1080/21670811.2014.884344
(18) Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann,
& M. Taddicken (Eds.), Digitale Methoden in der Kommunikationswissenschaft (Vol. 2, pp. 87–111). Berlin: digitalcommunicationresearch.de. Retrieved
from http://nbn-resolving.de/http://dx.doi.org/10.17174/dcr.v2.5
(3) Baack, S. (2013). A New Style of News Reporting: Wikileaks and Data-driven Journalism. In B. Rambatan & J. Johanssen (Eds.), Cyborg Subjects: Discourses on
Digital Culture (pp. 113–122). CreateSpace Independent Publishing. Retrieved from http://www.ssoar.info/ssoar/handle/document/40025
(26) Bakker, P. (2014). Mr. Gates Returns: Curation, Community Management and Other New Roles for Journalists. Journalism Studies, 15(5), 596–606. http:
//doi.org/10.1080/1461670X.2014.901783
(31) Daniel, A., & Flew, T. (2010). The Guardian Reportage of the UK MP Expenses Scandal: A Case Study of Computational Journalism. In Record of the
Communications Policy and Research Forum 2010 (pp. 186–194). Sydney: Network Insight. Retrieved from http://eprints.qut.edu.au/39358/
(35) De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for Data Journalism: A Qualitative Assessment of the Anecdotal Take-
up of Data Journalism in French-Speaking Belgium. Digital Journalism, 3(3), 432–446. http://doi.org/10.1080/21670811.2014.976415
(1) Diakopoulos, N. (2015). Algorithmic Accountability: Journalistic Investigation of Computational Power Structures. Digital Journalism, 3(3), 398–415. http:
//doi.org/10.1080/21670811.2014.976411
(21) Dick, M. (2014). Interactive Infographics and News Values. Digital Journalism, 2(4), 490–506. http://doi.org/10.1080/21670811.2013.841368
(19) Felle, T. (2015). Digital Watchdogs? Data Reporting and the News Media’s Traditional “Fourth Estate” Function. Journalism, 1464884915593246. http:
//doi.org/10.1177/1464884915593246
Corpus 1/3
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
55
(14) Fink, K., & Anderson, C. W. (2015). Data Journalism in the United States: Beyond the “Usual Suspects.” Journalism Studies, 16(4), 467–481. http://doi.
org/10.1080/1461670X.2014.939852
(27) Garrison, B. (1999). Newspaper Size as a Factor in Use of Computer-Assisted Reporting. Newspaper Research Journal, 20(3). Retrieved from http://com.
miami.edu/car/baltimore1.htm
(22)Gynnild, A. (2014). Journalism Innovation Leads to Innovation Journalism: The Impact of Computational Exploration on Changing Mindsets. Journalism, 15
(6), 713–730. http://doi.org/10.1177/1464884913486393
(7) Hannaford, L. (2015). Computational Journalism in the UK Newsroom: Hybrids or Specialists? Journalism Education, 4(1), 6–21. http://doi.org/http://hdl.
handle.net/2173/575953
(30) Howard, A. B. (2014). The Art and Science of Data-Driven Journalism (pp. 1–144). New York: Tow Center for Digital Journalism. Retrieved from http:
//towcenter.org/wp-content/uploads/2014/05/Tow-Center-Data-Driven-Journalism.pdf
(8) Hullman, J., Diakopoulos, N., Momeni, E., & Adar, E. (2015). Content, Context, and Critique: Commenting on a Data Visualization Blog. In Proceedings of the
18th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 1170–1175). New York: ACM. http://doi.org/10.1145/2675133.
2675207
(6) Karlsen, J., & Stavelin, E. (2014). Computational Journalism in Norwegian Newsrooms. Journalism Practice, 8(1), 34–48. http://doi.org/10.1080/17512786.
2013.813190
(13) Knight, M. (2015). Data Journalism in the UK: A Preliminary Analysis of Form and Content. Journal of Media Practice, 16(1), 55–72. http://doi.org/10.
1080/14682753.2015.1015801
(33) Koch, C., & Stollorz, V. (2014). Transparenz oder Konfusion? Methodik und empirische Basis des Datenjournalismus in der Medizin. In V. Lilienthal, D.
Reineck, & T. Schnedler (Eds.), Qualität im Gesundheitsjournalismus (pp. 367–387). Wiesbaden: Springer Fachmedien. Retrieved from http://link.springer.
com/chapter/10.1007/978-3-658-02427-7_21
(28) Lewis, S. C., & Usher, N. (2013). Open Source and Journalism: Toward New Frameworks for Imagining News Innovation. Media, Culture and Society, 35(5),
602–619. http://doi.org/10.1177/0163443713485494
(5) Lewis, S. C., & Usher, N. (2014). Code, Collaboration, and the Future of Journalism: A Case Study of the Hacks/Hackers Global Network. Digital Journalism, 2
(3), 383–393. http://doi.org/10.1080/21670811.2014.895504
(29) Lugo-Ocando, J., & Brandão, R. F. (2015). Stabbing News: Articulating Crime Statistics in the Newsroom. Journalism Practice, 9, 1–15. http://doi.org/10.
1080/17512786.2015.1058179
Corpus 2/3
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
56
(10) Parasie, S. (2015). Data-driven Revelation? Epistemological Tensions in Investigative Journalism in the Age of “Big Data.” Digital Journalism, 3(3), 364–380.
http://doi.org/10.1080/21670811.2014.976408
(9) Parasie, S., & Dagiral, E. (2013). Data-driven Journalism and the Public Good: “Computer-assisted-reporters” and “Programmer-journalists” in Chicago. New
Media & Society, 15(6), 853–871. http://doi.org/http://dx.doi.org/10.1177/1461444812463345
(32) Radchenko, I., & Sakoyan, A. (2014). The View on Open Data and Data Journalism: Cases, Educational Resources and Current Trends. In D. I. Ignatov, M. Y.
Khachay, A. Panchenko, N. Konstantinova, & R. E. Yavorsky (Eds.), Analysis of Images, Social Networks and Texts (pp. 47–54). Cham: Springer International
Publishing. Retrieved from http://link.springer.com/chapter/10.1007/978-3-319-12580-0_4
(25) Uskali, T., & Kuutti, H. (2015). Models and Streams of Data Journalism. The Journal of Media Innovations, 2(1), 77. http://doi.org/10.5617/jmi.v2i1.882
(24) Veglis, A. (2005). Modeling Computer Supported Collaborative Work in a Newspaper Editorial Process. In Proceedings of the 9th WSEAS International
Conference on Computers (pp. 1–6). Stevens Point: World Scientific and Engineering Academy and Society (WSEAS). Retrieved from http://dl.acm.
org/citation.cfm?id=1369599.1369726
(16) Weber, W., & Rall, H. (2012). Data Visualization in Online Journalism and Its Implications for the Production Process. In 2012 16th International Conference
on Information Visualisation (IV) (pp. 349–356). Montpellier: IEEE. http://doi.org/10.1109/IV.2012.65
(36) Weber, W., & Rall, H. (2013). “We Are Journalists”: Production Practices, Attitudes and a Case Study of the New York Times Newsroom. In W. Weber, M.
Burmester, & R. Tille (Eds.), Interaktive Infografiken (pp. 161–172). Berlin: Springer. Retrieved from http://link.springer.com/chapter/10.1007/978-3-642-
15453-9_9
(17) Weinacht, S., & Spiller, R. (2014). Datenjournalismus in Deutschland: Eine explorative Untersuchung zu Rollenbildern von Datenjournalisten. Publizistik, 59
(4), 411–433. http://doi.org/10.1007/s11616-014-0213-5
(20) Young, M. L., & Hermida, A. (2015). From Mr. and Mrs. Outlier to Central Tendencies: Computational Journalism and Crime Reporting at the Los Angeles
Times. Digital Journalism, 3(3), 381–397. http://doi.org/10.1080/21670811.2014.976409
Corpus 3/3
@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
MEDIA & DESIGN
57
Explore the (preliminary) corpus
online – it's a protoype:
http://tinyurl.com/valid-survey
http://validproject.at
Julian Ausserhofer1,2
,
@julauss
julian.ausserhofer@fh-joanneum.at
Robert Gutounig1
&
@sextus_empirico
robert.gutounig@fh-joanneum.at
Michael Oppermann2
@oppermann_m
michael.oppermann@univie.ac.at
1: FH Joanneum University of Applied Sciences Graz
2: University of Vienna
Dubrovnik Media Days, 31.10.15
#DuMD2015 & #DUMediaDays

Weitere ähnliche Inhalte

Ähnlich wie News Production Workflows in Data- driven, Algorithmic Journalism: A Systematic Literature Review

"Who is this redchanit?" Applying digital methods for issue mapping to one we...
"Who is this redchanit?" Applying digital methods for issue mapping to one we..."Who is this redchanit?" Applying digital methods for issue mapping to one we...
"Who is this redchanit?" Applying digital methods for issue mapping to one we...Jean Burgess
 
Using Graphs to Enable National-Scale Analytics
Using Graphs to Enable National-Scale AnalyticsUsing Graphs to Enable National-Scale Analytics
Using Graphs to Enable National-Scale AnalyticsNeo4j
 
Big data impact on society: a research roadmap for Europe (BYTE project resea...
Big data impact on society: a research roadmap for Europe (BYTE project resea...Big data impact on society: a research roadmap for Europe (BYTE project resea...
Big data impact on society: a research roadmap for Europe (BYTE project resea...Anna Fensel
 
Social media data stewardship: The ethics of social media data use for research
Social media data stewardship: The ethics of social media data use for researchSocial media data stewardship: The ethics of social media data use for research
Social media data stewardship: The ethics of social media data use for researchToronto Metropolitan University
 
Presentation data mining
Presentation data miningPresentation data mining
Presentation data miningAbdul Haseeb
 
Big data presentation for University of Reykjavik, Iceland, March 22
Big data presentation for University of Reykjavik, Iceland, March 22 Big data presentation for University of Reykjavik, Iceland, March 22
Big data presentation for University of Reykjavik, Iceland, March 22 Thorhildur Jetzek, Ph.D.
 
Big Data in NATO and Your Role
Big Data in NATO and Your RoleBig Data in NATO and Your Role
Big Data in NATO and Your RoleJay Gendron
 
Easy Data, Hard Data? Twitter Research and the Politics of Data Access
Easy Data, Hard Data? Twitter Research and the Politics of Data AccessEasy Data, Hard Data? Twitter Research and the Politics of Data Access
Easy Data, Hard Data? Twitter Research and the Politics of Data AccessAxel Bruns
 
New Forms of Data for e-Research
New Forms of Data for e-ResearchNew Forms of Data for e-Research
New Forms of Data for e-ResearchDavid De Roure
 
Towards a Community-driven Data Science Body of Knowledge – Data Management S...
Towards a Community-driven Data Science Body of Knowledge – Data Management S...Towards a Community-driven Data Science Body of Knowledge – Data Management S...
Towards a Community-driven Data Science Body of Knowledge – Data Management S...Research Data Alliance
 
Big data for situation awareness and decision making
Big data for situation awareness and decision makingBig data for situation awareness and decision making
Big data for situation awareness and decision makingPaloma Diaz
 
'Research hacks 23 ways to communicate and showcase yourself whilst working...
'Research hacks   23 ways to communicate and showcase yourself whilst working...'Research hacks   23 ways to communicate and showcase yourself whilst working...
'Research hacks 23 ways to communicate and showcase yourself whilst working...Andy Tattersall
 

Ähnlich wie News Production Workflows in Data- driven, Algorithmic Journalism: A Systematic Literature Review (20)

"Who is this redchanit?" Applying digital methods for issue mapping to one we...
"Who is this redchanit?" Applying digital methods for issue mapping to one we..."Who is this redchanit?" Applying digital methods for issue mapping to one we...
"Who is this redchanit?" Applying digital methods for issue mapping to one we...
 
Using Graphs to Enable National-Scale Analytics
Using Graphs to Enable National-Scale AnalyticsUsing Graphs to Enable National-Scale Analytics
Using Graphs to Enable National-Scale Analytics
 
Big data impact on society: a research roadmap for Europe (BYTE project resea...
Big data impact on society: a research roadmap for Europe (BYTE project resea...Big data impact on society: a research roadmap for Europe (BYTE project resea...
Big data impact on society: a research roadmap for Europe (BYTE project resea...
 
Digital Methods by Richard Rogers
Digital Methods by Richard RogersDigital Methods by Richard Rogers
Digital Methods by Richard Rogers
 
Social media data stewardship: The ethics of social media data use for research
Social media data stewardship: The ethics of social media data use for researchSocial media data stewardship: The ethics of social media data use for research
Social media data stewardship: The ethics of social media data use for research
 
Math in data
Math in dataMath in data
Math in data
 
Unpacking Digital Methods
Unpacking Digital MethodsUnpacking Digital Methods
Unpacking Digital Methods
 
Presentation data mining
Presentation data miningPresentation data mining
Presentation data mining
 
Big data presentation for University of Reykjavik, Iceland, March 22
Big data presentation for University of Reykjavik, Iceland, March 22 Big data presentation for University of Reykjavik, Iceland, March 22
Big data presentation for University of Reykjavik, Iceland, March 22
 
Big Data in NATO and Your Role
Big Data in NATO and Your RoleBig Data in NATO and Your Role
Big Data in NATO and Your Role
 
Easy Data, Hard Data? Twitter Research and the Politics of Data Access
Easy Data, Hard Data? Twitter Research and the Politics of Data AccessEasy Data, Hard Data? Twitter Research and the Politics of Data Access
Easy Data, Hard Data? Twitter Research and the Politics of Data Access
 
Carpenter: Lets remove alt from altmetrics - UKSG Plenary
Carpenter: Lets remove alt from altmetrics - UKSG PlenaryCarpenter: Lets remove alt from altmetrics - UKSG Plenary
Carpenter: Lets remove alt from altmetrics - UKSG Plenary
 
Rdaeu russia_fg_1_july2014_final
Rdaeu  russia_fg_1_july2014_finalRdaeu  russia_fg_1_july2014_final
Rdaeu russia_fg_1_july2014_final
 
Closing (D2L13 Insight@DCU Machine Learning Workshop 2017)
Closing (D2L13 Insight@DCU Machine Learning Workshop 2017)Closing (D2L13 Insight@DCU Machine Learning Workshop 2017)
Closing (D2L13 Insight@DCU Machine Learning Workshop 2017)
 
New Forms of Data for e-Research
New Forms of Data for e-ResearchNew Forms of Data for e-Research
New Forms of Data for e-Research
 
Carpenter - Lets remove "alt" from altmetrics - ER&L Presentation
Carpenter - Lets remove "alt" from altmetrics - ER&L PresentationCarpenter - Lets remove "alt" from altmetrics - ER&L Presentation
Carpenter - Lets remove "alt" from altmetrics - ER&L Presentation
 
Towards a Community-driven Data Science Body of Knowledge – Data Management S...
Towards a Community-driven Data Science Body of Knowledge – Data Management S...Towards a Community-driven Data Science Body of Knowledge – Data Management S...
Towards a Community-driven Data Science Body of Knowledge – Data Management S...
 
Big data for situation awareness and decision making
Big data for situation awareness and decision makingBig data for situation awareness and decision making
Big data for situation awareness and decision making
 
'Research hacks 23 ways to communicate and showcase yourself whilst working...
'Research hacks   23 ways to communicate and showcase yourself whilst working...'Research hacks   23 ways to communicate and showcase yourself whilst working...
'Research hacks 23 ways to communicate and showcase yourself whilst working...
 
Lecture rm 2
Lecture rm 2Lecture rm 2
Lecture rm 2
 

Mehr von Julian Ausserhofer

Digital Methods im Journalismus: Workflows in datenjournalistischen Projekten
Digital Methods im Journalismus: Workflows in datenjournalistischen ProjektenDigital Methods im Journalismus: Workflows in datenjournalistischen Projekten
Digital Methods im Journalismus: Workflows in datenjournalistischen ProjektenJulian Ausserhofer
 
Digitale Objekte in digitalen Öffentlichkeiten: Die Rolle von Hashtags in pol...
Digitale Objekte in digitalen Öffentlichkeiten: Die Rolle von Hashtags in pol...Digitale Objekte in digitalen Öffentlichkeiten: Die Rolle von Hashtags in pol...
Digitale Objekte in digitalen Öffentlichkeiten: Die Rolle von Hashtags in pol...Julian Ausserhofer
 
Political Discourses on Twitter: Networking Topics, Objects and People
Political Discourses on Twitter: Networking Topics, Objects and PeoplePolitical Discourses on Twitter: Networking Topics, Objects and People
Political Discourses on Twitter: Networking Topics, Objects and PeopleJulian Ausserhofer
 
Tagespolitik auf Twitter: Themen und Strukturen einer politischen Twittersphäre
Tagespolitik auf Twitter: Themen und Strukturen einer politischen TwittersphäreTagespolitik auf Twitter: Themen und Strukturen einer politischen Twittersphäre
Tagespolitik auf Twitter: Themen und Strukturen einer politischen TwittersphäreJulian Ausserhofer
 
Fact Checking on Social Media: Tales and Hacks
Fact Checking on Social Media: Tales and HacksFact Checking on Social Media: Tales and Hacks
Fact Checking on Social Media: Tales and HacksJulian Ausserhofer
 
Social Media, politische Partizipation & Mobilisierung
Social Media, politische Partizipation & MobilisierungSocial Media, politische Partizipation & Mobilisierung
Social Media, politische Partizipation & MobilisierungJulian Ausserhofer
 
Identität im Social Media
 „Sharing“: Authentizität als Präsentation 
multip...
Identität im  Social Media
 „Sharing“: Authentizität als Präsentation 
multip...Identität im  Social Media
 „Sharing“: Authentizität als Präsentation 
multip...
Identität im Social Media
 „Sharing“: Authentizität als Präsentation 
multip...Julian Ausserhofer
 
Vom Infodschungel zum digitalen Nutzgarten
Vom Infodschungel zum digitalen NutzgartenVom Infodschungel zum digitalen Nutzgarten
Vom Infodschungel zum digitalen NutzgartenJulian Ausserhofer
 
Datenjournalismus: Meilensteine und Charakteristika
Datenjournalismus: Meilensteine und CharakteristikaDatenjournalismus: Meilensteine und Charakteristika
Datenjournalismus: Meilensteine und CharakteristikaJulian Ausserhofer
 

Mehr von Julian Ausserhofer (10)

Digital Methods im Journalismus: Workflows in datenjournalistischen Projekten
Digital Methods im Journalismus: Workflows in datenjournalistischen ProjektenDigital Methods im Journalismus: Workflows in datenjournalistischen Projekten
Digital Methods im Journalismus: Workflows in datenjournalistischen Projekten
 
Digitale Objekte in digitalen Öffentlichkeiten: Die Rolle von Hashtags in pol...
Digitale Objekte in digitalen Öffentlichkeiten: Die Rolle von Hashtags in pol...Digitale Objekte in digitalen Öffentlichkeiten: Die Rolle von Hashtags in pol...
Digitale Objekte in digitalen Öffentlichkeiten: Die Rolle von Hashtags in pol...
 
Political Discourses on Twitter: Networking Topics, Objects and People
Political Discourses on Twitter: Networking Topics, Objects and PeoplePolitical Discourses on Twitter: Networking Topics, Objects and People
Political Discourses on Twitter: Networking Topics, Objects and People
 
The Relevance of Algorithms
The Relevance of AlgorithmsThe Relevance of Algorithms
The Relevance of Algorithms
 
Tagespolitik auf Twitter: Themen und Strukturen einer politischen Twittersphäre
Tagespolitik auf Twitter: Themen und Strukturen einer politischen TwittersphäreTagespolitik auf Twitter: Themen und Strukturen einer politischen Twittersphäre
Tagespolitik auf Twitter: Themen und Strukturen einer politischen Twittersphäre
 
Fact Checking on Social Media: Tales and Hacks
Fact Checking on Social Media: Tales and HacksFact Checking on Social Media: Tales and Hacks
Fact Checking on Social Media: Tales and Hacks
 
Social Media, politische Partizipation & Mobilisierung
Social Media, politische Partizipation & MobilisierungSocial Media, politische Partizipation & Mobilisierung
Social Media, politische Partizipation & Mobilisierung
 
Identität im Social Media
 „Sharing“: Authentizität als Präsentation 
multip...
Identität im  Social Media
 „Sharing“: Authentizität als Präsentation 
multip...Identität im  Social Media
 „Sharing“: Authentizität als Präsentation 
multip...
Identität im Social Media
 „Sharing“: Authentizität als Präsentation 
multip...
 
Vom Infodschungel zum digitalen Nutzgarten
Vom Infodschungel zum digitalen NutzgartenVom Infodschungel zum digitalen Nutzgarten
Vom Infodschungel zum digitalen Nutzgarten
 
Datenjournalismus: Meilensteine und Charakteristika
Datenjournalismus: Meilensteine und CharakteristikaDatenjournalismus: Meilensteine und Charakteristika
Datenjournalismus: Meilensteine und Charakteristika
 

Kürzlich hochgeladen

complaint-ECI-PM-media-1-Chandru.pdfra;;prfk
complaint-ECI-PM-media-1-Chandru.pdfra;;prfkcomplaint-ECI-PM-media-1-Chandru.pdfra;;prfk
complaint-ECI-PM-media-1-Chandru.pdfra;;prfkbhavenpr
 
Experience the Future of the Web3 Gaming Trend
Experience the Future of the Web3 Gaming TrendExperience the Future of the Web3 Gaming Trend
Experience the Future of the Web3 Gaming TrendFabwelt
 
57 Bidens Annihilation Nation Policy.pdf
57 Bidens Annihilation Nation Policy.pdf57 Bidens Annihilation Nation Policy.pdf
57 Bidens Annihilation Nation Policy.pdfGerald Furnkranz
 
15042024_First India Newspaper Jaipur.pdf
15042024_First India Newspaper Jaipur.pdf15042024_First India Newspaper Jaipur.pdf
15042024_First India Newspaper Jaipur.pdfFIRST INDIA
 
16042024_First India Newspaper Jaipur.pdf
16042024_First India Newspaper Jaipur.pdf16042024_First India Newspaper Jaipur.pdf
16042024_First India Newspaper Jaipur.pdfFIRST INDIA
 
IndiaWest: Your Trusted Source for Today's Global News
IndiaWest: Your Trusted Source for Today's Global NewsIndiaWest: Your Trusted Source for Today's Global News
IndiaWest: Your Trusted Source for Today's Global NewsIndiaWest2
 
Global Terrorism and its types and prevention ppt.
Global Terrorism and its types and prevention ppt.Global Terrorism and its types and prevention ppt.
Global Terrorism and its types and prevention ppt.NaveedKhaskheli1
 
Rohan Jaitley: Central Gov't Standing Counsel for Justice
Rohan Jaitley: Central Gov't Standing Counsel for JusticeRohan Jaitley: Central Gov't Standing Counsel for Justice
Rohan Jaitley: Central Gov't Standing Counsel for JusticeAbdulGhani778830
 

Kürzlich hochgeladen (8)

complaint-ECI-PM-media-1-Chandru.pdfra;;prfk
complaint-ECI-PM-media-1-Chandru.pdfra;;prfkcomplaint-ECI-PM-media-1-Chandru.pdfra;;prfk
complaint-ECI-PM-media-1-Chandru.pdfra;;prfk
 
Experience the Future of the Web3 Gaming Trend
Experience the Future of the Web3 Gaming TrendExperience the Future of the Web3 Gaming Trend
Experience the Future of the Web3 Gaming Trend
 
57 Bidens Annihilation Nation Policy.pdf
57 Bidens Annihilation Nation Policy.pdf57 Bidens Annihilation Nation Policy.pdf
57 Bidens Annihilation Nation Policy.pdf
 
15042024_First India Newspaper Jaipur.pdf
15042024_First India Newspaper Jaipur.pdf15042024_First India Newspaper Jaipur.pdf
15042024_First India Newspaper Jaipur.pdf
 
16042024_First India Newspaper Jaipur.pdf
16042024_First India Newspaper Jaipur.pdf16042024_First India Newspaper Jaipur.pdf
16042024_First India Newspaper Jaipur.pdf
 
IndiaWest: Your Trusted Source for Today's Global News
IndiaWest: Your Trusted Source for Today's Global NewsIndiaWest: Your Trusted Source for Today's Global News
IndiaWest: Your Trusted Source for Today's Global News
 
Global Terrorism and its types and prevention ppt.
Global Terrorism and its types and prevention ppt.Global Terrorism and its types and prevention ppt.
Global Terrorism and its types and prevention ppt.
 
Rohan Jaitley: Central Gov't Standing Counsel for Justice
Rohan Jaitley: Central Gov't Standing Counsel for JusticeRohan Jaitley: Central Gov't Standing Counsel for Justice
Rohan Jaitley: Central Gov't Standing Counsel for Justice
 

News Production Workflows in Data- driven, Algorithmic Journalism: A Systematic Literature Review

  • 1. MEDIA & DESIGN News Production Workflows in Data- driven, Algorithmic Journalism A Systematic Literature Review Julian Ausserhofer1,2 , @julauss Robert Gutounig1 & @sextus_empirico Michael Oppermann2 @oppermann_m 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays
  • 2. MEDIA & DESIGN *Tool evaluation partners * * * Supported by:
  • 4. MEDIA & DESIGN #DuMD2015 & #DUMediaDays @julauss, @sextus_empirico & @oppermann_m 4 "experimental use of algorithms, data and social science methods" in journalism (Gynnild, 2014, p. 715)
  • 5. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 5 (Cox, 2000) http://www.computerhistory.org/revolution/early-computer-companies/5/102/445 (Meyer, 1973) 1952 #DuMD2015 & #DUMediaDays Computational methods in journalism
  • 6. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 6 (Garrison, 1996) #DuMD2015 & #DUMediaDays
  • 7. MEDIA & DESIGN The datafication of journalism @julauss, @sextus_empirico & @oppermann_m 7 "datafication" of society (Mayer-Schönberger & Cukier, 2013) "increased focus on measurement, outcomes assessment" (Anderson, 2015, p. 363) "journalism's quantitative turn" (Coddington, 2015) … has also affected the "infrastructures of journalism" (Russ-Mohl, 2006, p.199) #DuMD2015 & #DUMediaDays
  • 8. MEDIA & DESIGN Research literature on data journalism @julauss, @sextus_empirico & @oppermann_m 8 "internalist tendencies at [... the] early stage of academic research" (Anderson, 2013, p. 1007) ↓ "an explosion in data journalism-oriented scholarship" (Fink & Anderson, 2015, p. 476)* "rapidly growing body" of scientific studies (Lewis, 2015, p. 322)* *cited via (Loosen, Reimer & Schmidt, 2015, p. 2) #DuMD2015 & #DUMediaDays
  • 9. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 9 In what way have journalistic workflows and routines changed due to the broad introduction of data in the newsroom? How has this impacted journalistic norms and ethics, and the skill requirements for journalists? #DuMD2015 & #DUMediaDays Research questions
  • 11. MEDIA & DESIGN #DuMD2015 & #DUMediaDays @julauss, @sextus_empirico & @oppermann_m 11 Systematic literature review
  • 12. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 12 "to develop insights, critical reflections, future research paths and research questions" (Massaro, Dumay & Guthrie, forthcoming) It adopts "a replicable, scientific and transparent process [...] that aims to minimize bias [...]" (Tranfield, Denyer & Smart, 2003) #DuMD2015 & #DUMediaDays Why a systematic literature review?
  • 13. MEDIA & DESIGN Undertaking a systematic literature review 13 Adapted from Massaro et al. (forthcoming) Own visualization @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Writing a literature review protocol Developing insights and critique through analyzing the dataset Developing future research paths and questions Determining the type of studies and carrying out a comprehensive literature search Coding dataDefining the questions that the literature review should answer
  • 14. MEDIA & DESIGN 14 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Determining the type of studies and carrying out a comprehensive literature search Defining the questions that the literature review should answer In what way have journalistic workflows and routines changed due to the broad introduction of data in the newsroom? How has this impacted journalistic norms and ethics, and the skill requirements for journalists?
  • 15. MEDIA & DESIGN 15 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search ● Empirical research on DDJ ● Social science focus, but open to other disciplines ● Published after 1995 Included Journal articles Book sections Conference papers Reports (from industry and research projects) PhD theses Not included Bachelor's and Master's theses Press reports Blog posts
  • 16. MEDIA & DESIGN 16 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search ● Preliminary search with “data-driven journalism” ● Extracting related terms from the keyword section of research papers
  • 17. MEDIA & DESIGN 17 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search Search terms algorithmic journalism computational journalism computer-assisted reporting data journalism data-driven journalism data-driven reporting database journalism datajournalism datenjournalismus quantitative journalism No search terms accountability journalism crowdsourced journalism dataviz datavis ddj drone journalism investigative journalism online journalism open journalism
  • 18. MEDIA & DESIGN 18 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search Scientific Databases ACM Digital Sowiport EBSCO Springer IEEE SpringerLink JSTOR Taylor & Francis Online ProQuest Web of Science Science Direct Wiley Scopus Google Scholar Sociological Abstracts
  • 19. MEDIA & DESIGN 19 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search 772 search results ↓ Assessment of title, abstract & keywords - by two independently working researchers (Thomas et al., 2004) ↓ 33 research publications
  • 20. MEDIA & DESIGN 20 Determining the type of studies and carrying out a comprehensive literature search Defining the questions that the literature review should answer Developing future research paths and questions Developing insights and critique through analyzing the dataset Coding data @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays qualitative coding computational analysis
  • 23. MEDIA & DESIGN 23 Research literature on workflows in DDJ Development of literature over time n=33 @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 24. MEDIA & DESIGN 24@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication type Parasie & Dagiral, 2013 Lewis & Usher, 2013 Diakopolus, 2015 Karlsen & Stavelin, 2014 n=33 node size = number of citations in Google Scholar
  • 25. MEDIA & DESIGN 25@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication year of referenced papers 1787-2015 n=1151 Constitution of the United States
  • 26. MEDIA & DESIGN 26@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication year of referenced papers 1999-2015 n=1004 2013: 176 cited papers
  • 27. MEDIA & DESIGN 27@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Most-cited references
  • 28. MEDIA & DESIGN 28@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays edges = 165 corpus publications, n=33 references, n = 70 size = nr. of citations from the corpus Research literature on workflows in DDJ Corpus and most-cited references The Journalist as Programmer Precision Journalism Accountability through Algorithm
  • 29. MEDIA & DESIGN Nr. of studies Survey 3 In-depth interviews 22 (Particpant) observation 4 Method(s) of data collection 29@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 30. MEDIA & DESIGN Geographical scope 30@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays Nr. of studies Nr. of studies United States 9 Multinational (>4) 2 United Kingdom 9 Norway 1 Germany 4 Netherlands 1 Sweden 2 Argentina 1 Switzerland 2 Austria 1 Finland 2 Belgium 1
  • 31. MEDIA & DESIGN Qualitative analysis - News Production Workflow
  • 32. MEDIA & DESIGN News production workflow Collecting Collecting digitalized information from ● publicly available data sets ● requested data ● own research or data collected through crowdsourcing ● automated research (scraping) ● leaked data (17; Loosen, 2015) 32@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 33. MEDIA & DESIGN Type of data source ● government or public bureau ● private corporation ● other organizations like NGOs, research institutes (28, Loosen, 2015) 33@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Collecting
  • 34. MEDIA & DESIGN Access to information in Norway and USA generally considered good (due to freedom of information legislation) Barriers to easy data access ● fees ● lacking competencies for exporting data at public offices ● the form data is often delivered (e.g. as PDF) ● lack of legal legal resources to get access to data that is denied to journalists in the first place (6; 14) 34@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Collecting
  • 35. MEDIA & DESIGN Selecting data Because data journalists don’t have much time in the newsroom, they prefer easy-accessible data, e.g. datasets from public bodies. They are very cautious when it comes to work with data from companies or private sources. (14) 35@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Collecting
  • 36. MEDIA & DESIGN Methods ● Journalists use statistical programs (e.g. MySQL, Excel, Access, SPSS, Google Docs, Maps, Google Refine, Google Fusion Tables etc.) ● Statistical methods and packages including cluster, network & regression analysis --> Journalists are quite cautious with it, since they consider this outside of their field of expertise. (6, 14) 36@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Analyzing
  • 37. MEDIA & DESIGN Results of DDJ research depend in increasing manner on (graphical) display of (automated) data analysis software Tools and programming languages ● Developers use Python, Ruby on Rails, JavaScript, HTML ● Used third-party software includes MySQL, Access, Excel, Caspio, Tableau, Arcview, TimelineSetter etc. (14, 17, 28) 37@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Building and visualizing
  • 38. MEDIA & DESIGN DDJ is connected to a special way of representation which consists of ● displaying key messages as graphics and/or ● building interactive web applications Storytelling is done through (1) interactive web applications with continuous update of data (2) interactive application based on a stable data set (3) continuously updated info graphic or (4) static infographics (17) 38@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Building and visualizing
  • 39. MEDIA & DESIGN ● Working alone at small and medium-sized newspapers, and part of teams at larger news organizations ● New connections between journalists and programmers ● Members of teams usually have complementary backgrounds (6, 9, 14, 18) 39@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Collaboration
  • 40. MEDIA & DESIGN ● Rise of "annotative journalism" ● Regarding the collaboration between newsroom and ICT- department in media corporations data journalists perceive the relation as troublesome. Rather data journalists try to bypass ICT department when installing extra software or setting up databases or servers (6, 28) 40@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Collaboration
  • 41. MEDIA & DESIGN ● The goal of DDJ activities is by definition the [very accurate] journalistic publication ● Publication of data sources and raw data following open data principles is mentioned too as an integral part of DDJ. (6, 17, Coddington, 2014) 41@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays News production workflow Publishing the product
  • 42. MEDIA & DESIGN Backgrounds of data journalists
  • 43. MEDIA & DESIGN Backgrounds of data journalists ● Data journalists are appx. 5 years younger than the average age of German journalists ● All of them have started a university program, 80% have also finished it ● There is no generalizable “data journalism” career path. Background is mostly either social science or informatics. ● Backgrounds include programming, design, typography, info graphics, usability, databases, Web and journalism ● USA: dominance of National Institute for Computer-Assisted Reporting (NICAR) (14, 17, 18; 6) 43@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 44. MEDIA & DESIGN Enabling and Hindering Factors
  • 45. MEDIA & DESIGN Enabling and indering factors Critical factors ● Skills of the involved journalists ● Support networks (internal and external) important for the development of skillsavailability of data ● Tools ● Legal resources ● Available resources (6, 14, 17) 45@julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 47. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 47#DuMD2015 & #DUMediaDays ● New kind of people arriving in the newsroom ● New skills needed in the newsroom ● Increase of collaborative newswork ● Shift to data-driven approaches (and epistemologies debates connected to that)
  • 48. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 48 Scope: ● Practices outside of Europe and North America ● The construction of "facts" ● Gender aspects Methods of data collection: ● Survey ● (Participant) observation ● Content analysis ● Digital methods #DuMD2015 & #DUMediaDays Research gaps
  • 49. MEDIA & DESIGN 49 Explore the (preliminary) corpus online – it's a protoype: http://tinyurl.com/valid-survey http://validproject.at Julian Ausserhofer1,2 , @julauss julian.ausserhofer@fh-joanneum.at Robert Gutounig1 & @sextus_empirico robert.gutounig@fh-joanneum.at Michael Oppermann2 @oppermann_m michael.oppermann@univie.ac.at 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays
  • 50. MEDIA & DESIGN Tools & Literature 50
  • 51. MEDIA & DESIGN 51 NVIVO [Computer Software]. (2015). Retrieved from http://www.qsrinternational.com/ Bostock, M., Ogievetsky, V., & Heer, J. (2011). D³ data-driven documents.Visualization and Computer Graphics, IEEE Transactions on, 17(12), 2301-2309. Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In Research and Advanced Technology for Digital Libraries (pp. 473-474). Berlin: Springer. Zotero [Computer Software]. (2015). Retrieved from http://zotero.org/ Tools & software libraries @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 52. MEDIA & DESIGN 52 Anderson, C. W. (2013). Towards a Sociology of Computational and Algorithmic Journalism. New Media & Society, 15(7), 1005–1021. http://doi.org/10.1177/1461444812465137 Anderson, C. W. (2015). Between the Unique and the Pattern: Historical Tensions in Our Understanding of Quantitative Journalism. Digital Journalism, 3(3), 349–363. http://doi.org/10.1080/21670811.2014.976407 Cox, M. (2000). The Development of Computer-assisted Reporting. Presented at the Newspaper Division, Association for Education in Journalism and Mass Communication, Southeast Colloquium, Chapel Hill, North Carolina. Fink, K., & Anderson, C. W. (2015). Data Journalism in the United States: Beyond the "Usual Suspects". Journalism Studies, 16(4), 467–481. http://doi.org/10.1080/1461670X.2014.939852 Garrison, B. (1996, September). Computer-Assisted Reporting: A National Use Update. Presented at the National Society of Professional Journalists Meeting, Washington, D.C. Lewis, S. C. (2015). Journalism In An Era Of Big Data. Digital Journalism, 3(3), 321–330. http://doi.org/10. 1080/21670811.2014.976399 Loosen, W., Reimer, J., & Schmidt, F. (2015). When Data Become News: A Content Analysis of Data Journalism Pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff. Mayer-Schönberger, V., & Cukier, K. (2013). Big Data: A Revolution That Will Transform How We Live, Work, and Think. Boston: Houghton Mifflin Harcourt. Primary Sources @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 53. MEDIA & DESIGN 53 Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the Shoulders of Giants: Undertaking a Structured Literature Review in Accounting. Accounting, Auditing & Accountability Journal. Meyer, P. (1973). Precision Journalism: A Reporter’s Introduction to Social Science Methods (2nd ed.). Oxford: Rowman & Littlefield. Russ-Mohl, S. (2006). The Economics of Journalism and the Challenge to Improve Journalism Quality: A Research Manifesto. Studies in Communication Sciences, 6(2), 189–208. Thomas, J., Harden, A., Oakley, A., Oliver, S., Sutcliffe, K., Rees, R., … Kavanagh, J. (2004). Integrating Qualitative Research with Trials in Systematic Reviews. BMJ : British Medical Journal, 328(7446), 1010–1012. http://doi.org/10.1136/bmj.328.7446.1010 Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. British Journal of Management, 14(3), 207–222. http://doi.org/10.1111/1467-8551.00375 Primary Sources @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 54. MEDIA & DESIGN 54 (34) Aitamurto, T., Sirkkunen, E., & Lehtonen, P. (2011). Trends in Data Journalism. Retrieved from http://virtual.vtt.fi/virtual/nextmedia/Deliverables-2011/D3. 2.1.2.B_Hyperlocal_Trends_In%20Data_Journalism.pdf (2) Ananny, M., & Crawford, K. (2015). A Liminal Press: Situating news app designers within a field of networked news production. Digital Journalism, 3(2), 192–208. http://doi.org/10.1080/21670811.2014.922322 (12) Appelgren, E., & Nygren, G. (2012). Data Journalism in Sweden – Opportunities and Challenges: A Case Study of Brottspejl at Sveriges Television (SVT). In C. Mills, M. Pidd, & E. Ward (Eds.), Proceedings of the Digital Humanities Congress 2012. Sheffield: HRI Online Publications. Retrieved from http://www. hrionline.ac.uk/openbook/chapter/dhc2012-appelgren (11) Appelgren, E., & Nygren, G. (2014). Data Journalism in Sweden: Introducing New Methods and Genres of Journalism into “Old” Organizations. Digital Journalism, 2(3), 394–405. http://doi.org/10.1080/21670811.2014.884344 (18) Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale Methoden in der Kommunikationswissenschaft (Vol. 2, pp. 87–111). Berlin: digitalcommunicationresearch.de. Retrieved from http://nbn-resolving.de/http://dx.doi.org/10.17174/dcr.v2.5 (3) Baack, S. (2013). A New Style of News Reporting: Wikileaks and Data-driven Journalism. In B. Rambatan & J. Johanssen (Eds.), Cyborg Subjects: Discourses on Digital Culture (pp. 113–122). CreateSpace Independent Publishing. Retrieved from http://www.ssoar.info/ssoar/handle/document/40025 (26) Bakker, P. (2014). Mr. Gates Returns: Curation, Community Management and Other New Roles for Journalists. Journalism Studies, 15(5), 596–606. http: //doi.org/10.1080/1461670X.2014.901783 (31) Daniel, A., & Flew, T. (2010). The Guardian Reportage of the UK MP Expenses Scandal: A Case Study of Computational Journalism. In Record of the Communications Policy and Research Forum 2010 (pp. 186–194). Sydney: Network Insight. Retrieved from http://eprints.qut.edu.au/39358/ (35) De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for Data Journalism: A Qualitative Assessment of the Anecdotal Take- up of Data Journalism in French-Speaking Belgium. Digital Journalism, 3(3), 432–446. http://doi.org/10.1080/21670811.2014.976415 (1) Diakopoulos, N. (2015). Algorithmic Accountability: Journalistic Investigation of Computational Power Structures. Digital Journalism, 3(3), 398–415. http: //doi.org/10.1080/21670811.2014.976411 (21) Dick, M. (2014). Interactive Infographics and News Values. Digital Journalism, 2(4), 490–506. http://doi.org/10.1080/21670811.2013.841368 (19) Felle, T. (2015). Digital Watchdogs? Data Reporting and the News Media’s Traditional “Fourth Estate” Function. Journalism, 1464884915593246. http: //doi.org/10.1177/1464884915593246 Corpus 1/3 @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 55. MEDIA & DESIGN 55 (14) Fink, K., & Anderson, C. W. (2015). Data Journalism in the United States: Beyond the “Usual Suspects.” Journalism Studies, 16(4), 467–481. http://doi. org/10.1080/1461670X.2014.939852 (27) Garrison, B. (1999). Newspaper Size as a Factor in Use of Computer-Assisted Reporting. Newspaper Research Journal, 20(3). Retrieved from http://com. miami.edu/car/baltimore1.htm (22)Gynnild, A. (2014). Journalism Innovation Leads to Innovation Journalism: The Impact of Computational Exploration on Changing Mindsets. Journalism, 15 (6), 713–730. http://doi.org/10.1177/1464884913486393 (7) Hannaford, L. (2015). Computational Journalism in the UK Newsroom: Hybrids or Specialists? Journalism Education, 4(1), 6–21. http://doi.org/http://hdl. handle.net/2173/575953 (30) Howard, A. B. (2014). The Art and Science of Data-Driven Journalism (pp. 1–144). New York: Tow Center for Digital Journalism. Retrieved from http: //towcenter.org/wp-content/uploads/2014/05/Tow-Center-Data-Driven-Journalism.pdf (8) Hullman, J., Diakopoulos, N., Momeni, E., & Adar, E. (2015). Content, Context, and Critique: Commenting on a Data Visualization Blog. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 1170–1175). New York: ACM. http://doi.org/10.1145/2675133. 2675207 (6) Karlsen, J., & Stavelin, E. (2014). Computational Journalism in Norwegian Newsrooms. Journalism Practice, 8(1), 34–48. http://doi.org/10.1080/17512786. 2013.813190 (13) Knight, M. (2015). Data Journalism in the UK: A Preliminary Analysis of Form and Content. Journal of Media Practice, 16(1), 55–72. http://doi.org/10. 1080/14682753.2015.1015801 (33) Koch, C., & Stollorz, V. (2014). Transparenz oder Konfusion? Methodik und empirische Basis des Datenjournalismus in der Medizin. In V. Lilienthal, D. Reineck, & T. Schnedler (Eds.), Qualität im Gesundheitsjournalismus (pp. 367–387). Wiesbaden: Springer Fachmedien. Retrieved from http://link.springer. com/chapter/10.1007/978-3-658-02427-7_21 (28) Lewis, S. C., & Usher, N. (2013). Open Source and Journalism: Toward New Frameworks for Imagining News Innovation. Media, Culture and Society, 35(5), 602–619. http://doi.org/10.1177/0163443713485494 (5) Lewis, S. C., & Usher, N. (2014). Code, Collaboration, and the Future of Journalism: A Case Study of the Hacks/Hackers Global Network. Digital Journalism, 2 (3), 383–393. http://doi.org/10.1080/21670811.2014.895504 (29) Lugo-Ocando, J., & Brandão, R. F. (2015). Stabbing News: Articulating Crime Statistics in the Newsroom. Journalism Practice, 9, 1–15. http://doi.org/10. 1080/17512786.2015.1058179 Corpus 2/3 @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 56. MEDIA & DESIGN 56 (10) Parasie, S. (2015). Data-driven Revelation? Epistemological Tensions in Investigative Journalism in the Age of “Big Data.” Digital Journalism, 3(3), 364–380. http://doi.org/10.1080/21670811.2014.976408 (9) Parasie, S., & Dagiral, E. (2013). Data-driven Journalism and the Public Good: “Computer-assisted-reporters” and “Programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. http://doi.org/http://dx.doi.org/10.1177/1461444812463345 (32) Radchenko, I., & Sakoyan, A. (2014). The View on Open Data and Data Journalism: Cases, Educational Resources and Current Trends. In D. I. Ignatov, M. Y. Khachay, A. Panchenko, N. Konstantinova, & R. E. Yavorsky (Eds.), Analysis of Images, Social Networks and Texts (pp. 47–54). Cham: Springer International Publishing. Retrieved from http://link.springer.com/chapter/10.1007/978-3-319-12580-0_4 (25) Uskali, T., & Kuutti, H. (2015). Models and Streams of Data Journalism. The Journal of Media Innovations, 2(1), 77. http://doi.org/10.5617/jmi.v2i1.882 (24) Veglis, A. (2005). Modeling Computer Supported Collaborative Work in a Newspaper Editorial Process. In Proceedings of the 9th WSEAS International Conference on Computers (pp. 1–6). Stevens Point: World Scientific and Engineering Academy and Society (WSEAS). Retrieved from http://dl.acm. org/citation.cfm?id=1369599.1369726 (16) Weber, W., & Rall, H. (2012). Data Visualization in Online Journalism and Its Implications for the Production Process. In 2012 16th International Conference on Information Visualisation (IV) (pp. 349–356). Montpellier: IEEE. http://doi.org/10.1109/IV.2012.65 (36) Weber, W., & Rall, H. (2013). “We Are Journalists”: Production Practices, Attitudes and a Case Study of the New York Times Newsroom. In W. Weber, M. Burmester, & R. Tille (Eds.), Interaktive Infografiken (pp. 161–172). Berlin: Springer. Retrieved from http://link.springer.com/chapter/10.1007/978-3-642- 15453-9_9 (17) Weinacht, S., & Spiller, R. (2014). Datenjournalismus in Deutschland: Eine explorative Untersuchung zu Rollenbildern von Datenjournalisten. Publizistik, 59 (4), 411–433. http://doi.org/10.1007/s11616-014-0213-5 (20) Young, M. L., & Hermida, A. (2015). From Mr. and Mrs. Outlier to Central Tendencies: Computational Journalism and Crime Reporting at the Los Angeles Times. Digital Journalism, 3(3), 381–397. http://doi.org/10.1080/21670811.2014.976409 Corpus 3/3 @julauss, @sextus_empirico & @oppermann_m#DuMD2015 & #DUMediaDays
  • 57. MEDIA & DESIGN 57 Explore the (preliminary) corpus online – it's a protoype: http://tinyurl.com/valid-survey http://validproject.at Julian Ausserhofer1,2 , @julauss julian.ausserhofer@fh-joanneum.at Robert Gutounig1 & @sextus_empirico robert.gutounig@fh-joanneum.at Michael Oppermann2 @oppermann_m michael.oppermann@univie.ac.at 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays