Presentation to CRC Mental Health Early Career Researcher Workshop, Melbourne 29.11.17 for @andsdata. Workshop title: A by-product of scientific training: We're all a little bit biased.

1. Richard Ferrers
Bias and the Data Life Cyle
Research Data Specialist, ANDS
29 Nov 2017
@valuemgmt v.4

2. Outline
 Bias: What is the problem? Where do I come from?
 The Code of Responsible Conduct for Research
 Types of Bias; cognitive, memory, emotional, cultural
 Data Approaches to engage with Bias
 Data in the research lifecycle
 Interesting notes on Bias; positivism vs pragmatism
 An exercise – qualitative research and bias

3. Introduction / Background
My perspective, My preferences / biases
My Data – google: Ferrers Figshare
https://figshare.com/authors/Richard_Ferrers/401493
My works: https://orcid.org/0000-0002-2923-9889
Topics: NBN, Value, Adoption / Diffusion of Innovation
My PhD: https://doi.org/10.6084/m9.figshare.680002.v9
An intepretive pragmatist social constructionist
Science/Bias is socially constructed.
Science as a contested discursive space.

5. NCRIS Research Infrastructure
•Data Storage for Communities
•Data (Cloud) Compute
•Research Data Management
“Massive datasets… [can] be assembled and explored [to] reveal… unsuspected
relationships… This data-led science is a promising source of knowledge”
Royal Society 2012 p.3

6. ANDS-Nectar-RDS
In 2017/18, we are working on four transformations
for Australian research:
 A world leading data advantage
 Accelerated innovation
 Collaboration for borderless research
 Enhanced translation of research
http://www.ands.org.au/about-us/ands-nectar-rds

7. The Australian National Data Service (ANDS) makes
Australia’s research data assets more valuable for
researchers, research institutions and the nation.
http://www.ands.org.au/about-us/what-we-do
people skills
services

8. The problem of bias
Community views, Scientific views

9. Quotes on Bias – a lay perspective
“It is useless to attempt to reason a man out of a thing he was
never reasoned into” Jonathan Swift
“It’s not at all hard to understand a person; it’s only hard to
listen without bias” Criss Jami, Killosophy
“We all see only what we are trained to see.” Robert Anton
Wilson, Masks of the Illuminati
“But I think that no matter how smart, people usually see
what they’re already looking for, that’s all.” Veronica Roth,
Allegiant (Source: GoodReads)

10. The answers you get depend on the questions you ask.p.139
“Normal science, the activity in which most scientists
inevitably spend almost all their time, is predicated on the
assumption that the scientific community know what the
world is like.” p.5
There are significant limits to what proponents of different
theories can communicate to one another; incommensurability (p.198)
For example, flat vs round earth; helio or geo-centric.
The Structure of Scientific Revolutions (1962/70) Citations:
>80k Excerpt
Thomas Kuhn - Paradigms

11. What’s the problem?
 Why most published research findings are false? PLoS
Medicine, Iaonnidis 2005, 8, 2, e124. (vs Goodman & Greenland)
 Publishing exciting, counter-intuitive findings
 How biased is science, really? Washington Post, 31.3.17
 Replicate 100 psychology studies, 36 get same results
 Let’s stop pretending like Science is perfect. Washington Post, 13.1.17
 Science 2015, 349, 6251, aac4716 DOI: 10.1126/science.aac4716
 Does Science have a replication crisis? (Maybe not ...)
Washington Post 9.3.16

12. Aust Code for Responsible Research (1.6)
 Intellectual honesty and integrity
 Scholarly and scientific rigour
 Instn climate for responsible and ethical behavior (1.1)
 Institutions provide induction, formal training and
continuing education for all research staff, inc. trainees
(1.3)
 Adopt methods appropriate for achieving the aims of
each research proprosal
 Conform to policies – institution -funder NHMRC 2007

13. Types of Bias
My perspective, My preferences / biases
Keen to speak to NBN customers on HFC,
get in touch if you are not happy
with your service #NBN (The Aust).
180,000 plus complaints about @TurnbullMalcolm
second rate #nbn means $30 billion waste and
no jobs for #gennbn #broadband #telecom @NBNCo
Assoc. Prof Mark Gregory, RMIT

14. What is bias?
 Iaonnidis (2005). PLoS Medicine “let us define bias as
the combination of various design, data, analysis and
presentation factors that tend to produce research
findings when they should not have been produced.”
 Fanelli et al. (2017) PNAS Bias patterns include: small
study effects, gray literature bias, decline effects, early
extreme, citation bias, industry bias
 Wikipedia – list of cognitive biases; heuristics (too
much; too fast), emotion, social influence (Nobel Prize
Economics – Kahneman) Royal Society. Video

15. List of Cognitive Biases
•Too much information
•Need to act fast
•Not enough meaning
•What to keep/delete
Wikipedia

16. Data approaches to engage with bias

17. Data in the Research Life Cycle
Data
Cite data
Data
Publish data
Data
Collect, store, analyse data
Data
Find data
Data
Plan for data
Source: Bournemouth University
RDA – the national
data catalogue
Re3data– the int’l
data catalogue
Unimelb checklist
Datacite stats
FAIR data checklist
eg Figshare
CRC Mental Health

18. Data driven solutions to bias
 TOP Guidelines “Reproducibility of research can be
improved by increasing transparency of the research
process and products… eight standards: citation, data,
analytic methods, research materials, design and
analysis, preregistration (study, analysis plan),
replication.” cos.io/TOP (Transparency and Openness)
 Publishing and describing your datasets – subject to
ethics, community consent, legal, privacy
 “Open enquiry is at the heart of the scientific
enterprise… permits others to identify errors” Royal
Soc.2012

19. ANDS resources on managing medical data
 Health and Medical webinar series
 Guide to publishing and safely sharing sensitive data
 Including informed consent
 De-identifying medical data
 Training – 10 Health and Medical Data Things
“I know the data is anonymised, but I want you to remember
That it’s my story – it’s about me, my life, my family
Researchers should honour that by making information
available about what the data is used for and what is found” A patient
Patient Views on Data Sharing – Anne McKenzie WA CCHRN

20. Other solutions to bias
 Peer Review
 Working in a team; induction, training, review
 Understanding your own assumptions – ontology and
epistemology; incommensurability
 “Open enquiry is at the heart of the scientific
enterprise… permits others to identify errors” Royal
Soc.2012
 Documenting your process, assumptions,
approaches

21. Notes on bias
How UQ Business trains management PhDs – my
experience

22. How the UQ Business School trains PhDs
 Positivism vs Interpretivism; social construction of
reality; ontology, epistemology, incommensurability
 Paradigm wars; qual vs quant; hard vs soft science
 Foucault 1969 – The Archaeology of Knowledge
 Berger and Luckmann 1966 – Social Construction of
Reality (citations: >39k)
 Glaser and Strauss 1967 - The Discovery of Grounded
Theory (citations: >78k) | Interpretive validity Sandberg (2005)
 Kuhn 1962 – The Structure of Scientific Revolutions

23. Data approaches to the Rsch Lifecycle
Activity – Review a study critically – find the biases
- review list of cognitive biases; believeable?
- review Washington Post articles; is there
bias?
- review Iaonnidis 2005; is he credible?
- review ANDS Guides on Sensitive data
sharing; Doable? 20mins, then 10min to discuss

24. With the exception of third party images or where otherwise indicated, this work is licensed under the Creative
Commons 4.0 International Attribution Licence.
ANDS, Nectar and RDS are supported by the Australian Government through the National Collaborative Research
Infrastructure Strategy Program (NCRIS).
Research Data Specialist, CRC Mental Health
ANDS Liaison
richard.ferrers@ands.org.au
+61 3 9902 0569 @valuemgmt
Richard Ferrers