Varsha Khodiyar, PhD
Data Curation Editor, Scientific Data
Nature Publishing Group
@varsha_khodiyar
@scientificdata
Tweet with #SDJPN16
Gaining credit for sharing research data
Data publishing with Scientific Data
RIKEN Center for Life Science Technologies 4th March 2016

My background
• Joined Scientific Data in October 2014
• Professional data curator since 2003
• PhD in Molecular Biology from the University of
Leicester
• Contributed to the Human Genome Project as
member of the Human Gene Nomenclature
Committee (HGNC)
• Gene Ontology curator for 8 years, at University
College London, UK
• 3 years of open data publishing experience
2

Why share research data?

Generating research data is expensive
Just 18.1% NIH grant applications funded in 2014*
• Hours spent writing grants?
• Hours spent reviewing grants?
Resources are finite/expensive
• Modified animals
• Specialized reagents
Time and effort taken in the laboratory to generate
good, valid data
* report.nih.gov/success_rates/Success_ByIC.cfm

Irreproducibility of published science
Figure 1 - Ioannidis JPA. et al. Repeatability of published microarray gene
expression analyses. Nature Genetics 41, 149–55 (2009) doi:10.1038/ng.295

Withholding data impacts on human health
Clinical study reports, detailed data and software code available at Dryad
Digital Repository doi:10.5061/dryad.bv8j6 and www.Study329.org

• Diversity of analyses and opinion
• New research
• testing of new hypotheses
• new analysis methods
• meta-analyses to create new
datasets
• studies on data collection methods
• Education of new researchers
• Increased return on investment in
research
Vickers AJ: Whose data set is it anyway? Sharing raw
data from randomized trials. Trials 2006, 7:15
Hrynaszkiewicz I, Altman DG: Towards agreement on
best practice for publishing raw clinical trial data.
Trials 2009, 10:17
Sharing data promotes

Researchers already share data
• Most researchers are sharing
data, and using the data of
others
• Direct contact between
researchers (on request) is a
common way of sharing data
• Repositories are second most
common method of sharing
Kratz and Strasser (2015) doi: 10.1371/journal.pone.0117619 9

Some problems…
• Sharing upon request relies heavily on trust
• Informally stored data associated with published works disappears at a
rate of ~17% per year (Vines et al. 2014; doi: 10.1016/j.cub.2013.11.014)
• Datasets not referenced in a manuscript are essentially invisible (a.k.a
“Dark data”)
• If data are available, they are often not interpretable or reusable
because sufficient detail is not included
• Data producers do not get appropriate credit for their work

10
www.nature.com/scientificdata

Credit – Scholarly credit for publishing data; all publications are indexed
and citeable.
Reuse – Standardized and detailed descriptions enables easier reuse of
published research data.
Quality – Rigorous peer-review on technical quality and reusability.
Editorial Board of experts in their field maintain community standards.
Discovery – Curated, machine-readable metadata for dataset discovery.
Validated links to published data in each article.
Open – Use of CC-BY licence for articles and CC0 for metadata. Promote
use of open licences for published data.
Service – Commitment to excellent service for authors and readers.

What is a Data Descriptor?

Data Descriptors have human and machine readable
components
13
Human readable
representation of
study
i.e. article (HTML &
PDF)
Human readable
representation of
study
i.e. article (HTML
& PDF)
Machine
readable
representation
of study
i.e. metadata

Synthesis
Analysis
Conclusions
What did I do to generate the data?
How was the data processed?
Where is the data?
Who did what and when?
Methods and technical analyses supporting the quality of the measurements.
Do not contain tests of new scientific hypotheses
Comparison of Data Descriptor to traditional article

What types of data can be published?
15
Decades
old
dataset
Standalone
dataset
Data that has been
used in an analysis
article
Large
consortium
dataset
Data from a
single
experiment
Data that the
researcher finds
valuable and that
others might find
useful too
Data associated
with a high impact
analysis article

When can a Data Descriptor be published?
16
After data
analysis has
been
published
Before analysis
has been
published
Authors not
intending to
analyse data
Data Descriptors can be
submitted and published
at any point in the
research workflow, i.e.
whenever it makes most
sense for your data
After data
analysis has
been
published
Before the
analysis has
been published
Publication
alongside analysis
article

Scientific Data accepts submissions from all quantitative
research disciplines
17

Helping authors find the right
place for their data

Scientific Data’s Repository List
Browse our recommended data repositories online.
• We currently list almost 80 repositories, across biological, medical,
physical and social sciences
• When required, we provide guidance to authors on the best place to
store their data
www.nature.com/sdata/data-policies/repositories

Generation of machine readable
metadata

• We want to capture metadata about the dataset being described in each Data
Descriptor
• The manuscript captures human readable metadata needed for data reuse
• The curated metadata records capture machine readable metadata needed for
machine based data discovery
Metadata at Scientific Data

ISA-Tab format for machine readable metadata
22
• Study workflow
• Key sample characteristics
needed for data discovery
• Relates samples to data files
• Shows location of dataset
• Uses controlled vocabularies
and ontologies (where
possible)

Use of community endorsed ontologies and controlled
vocabularies
23
Controlled vocabulary = list of standardized phrases of scientific concepts
Ontology = controlled vocabulary with defined relationships between terms

Structured Summary table from curated metadata
24
Investigation file
Study file
Sample characteristics reported in Structured Summary table:
Organism
Organism part
Cell line
Geographical location
Environment type

Viewing the metadata
25
1.
2.
3.

Metadata for data discovery
Search by:
• Data Repositories
• Experiment design
• Measurements made
• Technologies used
• Factor types
• Sample Characteristics
• Organism
• Environment types
• Geographic locations
scientificdata.isa-explorer.org

Citing Data

Citing my own data
1. In the
article text
2. In the Data
Citation section

Citing data I’ve reused
1. In the
article text
2. In the
References
section

Clinical researchers support sharing, but…
Rathi V, Dzara K, Gross CP, Hrynaszkiewicz I, Joffe S, Krumholz HM, Strait KM, Ross JS:
Sharing of clinical trial data among trialists: a cross sectional survey. BMJ 2012;345:e7570
• Sharing de-identified data via repositories should be
required (236 respondents, 74%)
• Investigators should share de-identified data on request
(229 respondents, 72%)

…clinical data producers have specific concerns
Rathi V, Dzara K, Gross CP, Hrynaszkiewicz I, Joffe S, Krumholz HM, Strait KM, Ross JS: Sharing of
clinical trial data among trialists: a cross sectional survey. BMJ 2012;345:e7570

Example initiatives for sharing clinical data
Yale Open Data Access (YODA) & Clinical Study Data
Request (CSDR) projects:
• Data Use Agreements (DUAs)
• Controlled access environment
• Scientific validity of reanalysis checked
• Independent governance
• Data anonymisation checks
http://yoda.yale.edu/
https://www.clinicalstudydatarequest.com/

Clinical data publication at Scientific Data
• Identify repositories able to archive clinical data
• Work with identified repositories to establish workflows for
peer review and publication, whilst maintaining patient
privacy
• Facilitate specialist peer review process for clinical data, for
example ensure peer reviewers have agreed to terms of data
use agreement
Hrynaszkiewicz, I., Khodiyar, V., Hufton, A. & Sansone, S. A. Publishing descriptions of non-
public clinical datasets: guidance for researchers, repositories, editors and funding
organisations. BioRxiv http://dx.doi.org/10.1101/021667 (2015).

A robust data-on-request workflow?

Published Data Descriptor with clinical data
Data Records
section details
how to access
the data

Links to restricted access data Data
Citations link
to repository
Data files
requiring
permission
to access
Freely
accessible
data files

Data Reuse stories

Data reuse by (some of) the same researchers
38

Data reuse by other researchers in the same field
39
“The Data Descriptor made it easier
to use the data, for me it was critical
that everything was there…all the
technical details like voxel size.”
Professor Daniele Marinazzo

According to Google
Scholar, cited 43 times!
(February 2016)
Data reuse and citation by researchers

41
www.bbc.co.uk/news/science-environment-33057402
Data reuse by the non-research community

Data reuse by the non-research community
42
http://www.nytimes.com/interactive/2014/12/30/science/history-of-ebola-in-24-outbreaks.html

Data Descriptors…
• …enable you to gain scholarly credit for your data gathering
efforts.
• …are human AND machine readable.
• …can be published with, or independently of, an analysis article.
• …can be published point in the research workflow.
• …allow the publication and discovery of clinical data, whilst
maintaining your patients privacy.
• …result in greater reuse and citation by fellow members of your
research community.
• …extend the impact of your research data by enabling access to
and reuse by the non-research community.
43

Get more
from
your data
Preserve
it
Encourage
reuse
Get credit
for it
Visit nature.com/sdata
Email scientificdata@nature.com
Tweet @ScientificData #SDJPN16