Presentation given at Open Science question and answer session hosted by the Institute for Quantitative Social Science (IQSS), and the Office for Scholarly Communication (OSC) at Harvard University, on July 16th 2014.

1. OPEN SCIENCE: YOUR QUESTIONS ANSWERED
Varsha Khodiyar
Data Publishing Manger, F1000Research
@vkf1000
Michael Markie
Associate Publisher, F1000Research
@mmmarksman
f1000research.com
@f1000research

2. OPEN SCIENCE PUBLISHING MEANS:
• Open access articles
• Open peer review
• Open data
• Open software
• Full research provenance
“Open science is the movement to make scientific
research, data and dissemination accessible to all
levels of an inquiring society, amateur or professional.”

3. ABOUT F1000RESEARCH – THE FIRST OPEN SCIENCE JOURNAL
The Seer of Science
Publishing
Science 4 October 2013:
Vol. 342 no. 6154 pp. 66-67
DOI: 10.1126/science.342.6154.66
http://www.sciencemag.org/conte
nt/342/6154/66.full.pdf
http://blog.f1000.com/2013/10/04/vitek-
tracz-science-interview/

4. WHAT IS F1000RESEARCH?
F1000Research is an open science journal for life scientists that accepts
all scientifically sound articles, ranging from single findings, case
reports, protocols, replications, and null or negative results to more
traditional articles.
Key features:
• Publication within a week
• Transparent, post-publication peer review
• All data included
• Accepts non-traditional article types

5. OPEN PEER REVIEW

6. TYPES OF PEER REVIEW
Time of review:
• Before publication: mediated by each individual journal
• Cascading review: reviews carried over to the next journal after rejection
• Third-party review: the peer review is no longer coupled to a journal.
• Post-publication peer review: journal publishes the article, then
reviewers look at it.
Transparency of review:
• Single-blind: the reviewer knows who the authors are, but the authors
don’t know who the reviewers are
• Double-blind: authors and reviewers are both anonymous
• Open peer review: all names are public.
See: http://www.britishecologicalsociety.org/publications/journals/ for examples of each

7. TRADITIONAL PEER REVIEW
Time of review:
• Before publication: mediated by each individual journal
• Cascading review: reviews carried over to the next journal after rejection
• Third-party review: the peer review is no longer coupled to a journal.
• Post-publication peer review: journal publishes the article, then
reviewers look at it.
Transparency of review:
• Single-blind: the reviewer knows who the authors are, but the authors
don’t know who the reviewers are
• Double-blind: authors and reviewers are both anonymous
• Open peer review: all names are public.
See: http://www.britishecologicalsociety.org/publications/journals/ for examples of each

8. TRADITIONAL PUBLISHING AND PEER REVIEW
TIMELINE

9. ISSUES WITH TRADITIONAL PEER REVIEW SYSTEM
• Lack of transparency
- Who are the reviewers?
- What happened with this paper
before it was accepted?
• Lack of accountability
- Anonymous reviews
- Editorial decisions may not reflect
reviews
• Inefficiency
- Re-reviewing the same work at
different journals
• Delays
• incidental (reviewing takes time)
• deliberate (reviewers delaying competitor papers)
Cartoon by Nick D Kim, strange-matter.net

10. A POST-PUBLICATION APPROACH TO PEER REVIEW

11. REFEREE SCORES
• Approved
• Approved with reservations
• Not approved
Articles with sufficient positive evaluations
are indexed in PubMed, Scopus, and Embase.
or
Minimal requirements for
indexing

12. OPEN PEER REVIEW AND OPEN COMMUNITIES
Referee reports and
other comments are
visible to anyone.
Community
input

13. BENEFITS OF TRANSPARENT REVIEW FOR AUTHORS
AND READERS
• Visible discussion between referees and authors (and editors) puts
paper in context.
• Referees are good at spotting broader significance of an article.
• Shows the back story of a paper. (e.g. Why did it take 3 rounds of
review?
• Authors can demonstrate that their paper was reviewed by top
people in their field.
• Reduces bias amongst referees
• Educational aspect of open peer review:
• Open referee reports can serve as examples.
• Demonstrates differences between reviewers

14. BENEFITS FOR REVIEWERS
• Get a DOI: Take credit for hard work
• Demonstrate experience as reviewer
• Shows reviewer’s informed opinion of the work as a peer in
the field, and where they thought it could be improved.
• Especially relevant in borderline cases, where an article just barely
passed review.
• Here’s what the community think: http://vimeo.com/99777547.

15. WHY SHARE DATA AND CODE?

16. WHY SHARE YOUR DATA?
Transparency and openness are cornerstones of the scientific method
“Not allowing reuse of
data is scientific
malpractice”
Royal Society; Science as
an open enterprise, Final
report 2012
http://royalsociety.org/about-us/history/

17. SHARING DATA CORRELATES WITH HIGHER CITATIONS
“We conclude there is a direct effect of third-party data reuse
that persists for years beyond the time when researchers
have published most of the papers reusing their own
data...We further conclude that...a substantial fraction of
archived datasets are reused, and that the intensity of
dataset reuse has been steadily increasing since 2003.”
Piowar HA., Vision TA. Data reuse and the open data citation
advantage. PeerJ (2013) doi: 10.7717/peerj.175

18. SHARING DATA ADDITIONALLY PROMOTES
• Diversity of analyses and opinion
• New research
• testing of new hypotheses
• new analysis methods
• meta-analyses to create new
datasets
• studies on data collection methods
• Reduction of error and fraud
• Education of new researchers
• Increased return on investment in
research

19. PUSH TOWARDS DATA SHARING

20. SHARING DATA ALLOWS REPLICATION
“[W]e evaluated the replication of data analyses in 18 articles
on microarray-based gene expression profiling published in
Nature Genetics in 2005–2006...We reproduced two analyses
in principle and six partially or with some discrepancies; ten
could not be reproduced. The main reason for failure to
reproduce was data unavailability.”
Ioannidis JPA. et al. Repeatability of
published microarray gene expression
analyses.
Nature Genetics 41, 149–55 (2009)

21. RESEARCH BECOMES HARDER TO ACCESS WITH AGE
“• We examined the availability of data from 516 studies between
2 and 22 years old
• The odds of a data set being reported as extant fell by 17% per
year
• Broken e-mails and obsolete storage devices were the main
obstacles to data sharing
• Policies mandating data archiving at publication are clearly
needed”
Vines TH. et al. The availability of research data declines rapidly
with article age. Curr Biol 24, 94–7 (2014)

22. MAKING DATA AND CODE TRULY OPEN

23. FULL DATA INTEGRATION WITH RESEARCH PAPERS

24. DATA ARTICLES
A dataset (or set of datasets) together with the associated
methods/protocol used to create the data. No analysis of the data,
results or conclusions should be included.
http://f1000research.com/author-guidelines#data-art-sub
“One goal we had for publishing this Data article in
F1000Research was to quickly share some of our ongoing
behavioral datasets in order to encourage collaboration
with others in the field.”
Donald Cooper, University of Colorado, Boulder
http://f1000research.com/articles/2-53/v1

25. SOFTWARE AVAILABILITY SECTION

26. DATA PLOTTING TOOL
http://blog.f1000research.com/2013/11/11/data-plotting/

27. DATA PLOTTING TOOL

28. ONGOING PROJECTS AT F1000RESEARCH
F1000Research is involved in discussions with institutional librarians, researchers and
other journals concerning data publication and sharing issues:
• Increasing the value of shared datasets (Force11, RDA)
• Tracking the usage of datasets using altmetrics
• Making data and software accessible (documented access route, or conditions of
access)
• Ensuring data and software is useable
• Facilitating appropriate recognition for scientists (e.g. citable peer review)
• Addressing data storage issues
• Repository accreditation (PREPARDE project)
• Data format and software depreciation