Jadranka Stojanovski presents PEERE and shares the editors' opinions on Open Peer Review (Croatian OA journals) | OSFair2017 Workshop Workshop title: Peer review at the crossroads Workshop overview: The workshop builds on the results of the OpenUp landscape scan and the OpenAIRE report on open peer review. The workshop has multiple purposes including (1) assessing existing and evolving methods and functions of alternative peer review mechanisms, (2) breaking down peer review into the basic processes to identify the benefits and challenges, and (3) identifying questions and issues that need further investigation. Group discussions will also touch upon issues such as the sustainability, long-term availability of alternative review tools, and their uptake by researchers, and the incorporation of these methods into institutional, national, funders’ and publishers’ policies. OpenUP and OpenAIRE are dedicated to engage with different (disciplinary, inter-disciplinary) research communities from the social sciences, life sciences, energy, arts and humanities to identify the requirements from the emerging trends as posed by Open Science and e-infrastructural interconnected environments. Both projects aim at developing a sustainable framework that is relevant for and responsive to the Open Science needs. DAY 3 - PARALLEL SESSION 6

1. PEERE - RESEARCH ON PEER REVIEW
EDITORS’ OPINIONS ON OPEN PEER REVIEW
(CROATIAN OA JOURNALS)
Jadranka Stojanovski
University of Zadar / Ruđer Bošković Institute
Open Science Fair 2017, 6-8 September, Athens, Greece

2. • Peer reviewers don’t agree much (1899 articles)
• Reviewers don’t reliably select highly cited articles (Bornmann &
Daniel Angew Chem 2008;47:7173-8)
• Peer review is not effective at detecting errors (Godlee et al. The
impact of blinding and masking on the quality of peer review. JAMA
1998;280:237-40)
• Rigorous peer-review has not prevented the publication of
fraudulent science in major journals
• slow, expensive, biased, unreliable, open to abuse (Wager & Jefferson,
Learned Pub 2001;14:257-63)
Open Science Fair 2017, 6-8 September, Athens, Greece
Liz Wager, PUBMET, Zadar, 2014

3. New Frontiers of Peer Review (PEERE)
• Trans-Domain COST Action TD1306
• 12/5/2014-11/5/2018
• Chair: Prof Flaminio SQUAZZONI
• http://www.peere.org/
• „to improve efficiency, transparency and accountability of peer
review through a trans-disciplinary, cross-sectorial
collaboration”
Open Science Fair 2017, 6-8 September, Athens, Greece

4. 0
50
100
150
200
250
300
350
1960 1970 1980 1990 2000 2010 2020
# papers on peer review (Scopus)
Open Science Fair 2017, 6-8 September, Athens, Greece

5. Objectives
• to analyse peer review in different scientific areas by integrating
quantitative and qualitative research;
• to evaluate implications of different models of peer review and
to explore new incentive structures, rules and measures;
• to involve science stakeholders in data sharing and testing
initiatives;
• to define collaboratively a joint research agenda that points to an
evidence-based peer review reform.
Open Science Fair 2017, 6-8 September, Athens, Greece

6. BENEFITS OF OPEN PEER REVIEW
• author can see who reviewed their work
• reviewer comments put paper in the context which is useful
additional information for readers
• reduces bias among reviewers
• more constructive reviews
• published report can serve as peer review examples for young
researchers
• shows the reviewer’s informed opinion on the work
• demonstrates experience of the reviewer
• can take credit for the work involved in conducting the review
Open Science Fair 2017, 6-8 September, Athens, Greece
Squazzone, 2016

7. OPEN PEER REVIEW RESEARCH
• The potential drawbacks of open peer review. A simulation
model of scientist behaviour (Squazzoni & Bianchi)
• Getting it transparent or keeping it obscure? Potential
implications of open peer review on scientist competition and
collaboration (Federico Bianchi and Flaminio Squazzoni)
• http://www.peere.org/dissemination/presentations/
Open Science Fair 2017, 6-8 September, Athens, Greece

8. Collaboration with big publishers
• PEERE Data Sharing Protocol: regulate data sharing on peer
review in all PEERE activities
• the document has been completed on 23 December 2016 with
Elsevier, Springer and Wiley representatives, and has been
signed by all partners on 1 March 2017
Open Science Fair 2017, 6-8 September, Athens, Greece

9. hrcak.srce.hr
Open Science Fair 2017, 6-8 September, Athens, Greece

10. THE SURVEY ON PEER REVIEW
Open Science Fair 2017, 6-8 September, Athens, Greece
441 OA journals
218 with peer
review
51
STM
78
SSH
39
questions

11. QUESTIONS
Information on their present peer review
process
• acceptance rate
• peer review type
• number of reviewers
• instructions for peer reviewers
• automated peer review tool
• timeliness
• international recommendations
• ethical issues
• etc.
Opinions about open peer review
• what is OPR?
• opinions on possible improved quality,
awards, acceptance, willingness of the
reviewers to disclose their identity,
bias, open discussion in the academic
community, objectivity, etc.
Open Science Fair 2017, 6-8 September, Athens, Greece

12. PEER REVIEW IN THE CROATIAN OA
JOURNALS (1)
• 58% employ peer reviewer from other institutions, 24% from the
journal’s institution
• 0% cascade peer review, 0% crowdsourcing peer review,
0% open peer review
• most journals practice „double blind” peer review
• 91% employ two or more peer reviewers per manuscript (mostly one
Croatian, and one from abroad)
• peer review process takes 120 days (mean)
Open Science Fair 2017, 6-8 September, Athens, Greece

13. PEER REVIEW IN THE CROATIAN OA
JOURNALS (2)
• editors are mostly satisfied with the quality of the submitted reviews,
and peer reviewers meet the deadlines
• it is sometimes difficult to find competent peer reviewers
• only 15% ask from the peer reviewers to declare conflict of interest
• only 40% do plagiarism check before peer review
• 45% have publicly available instructions for peer reviewers
• 43% explain major ethical issues in the instructions
Open Science Fair 2017, 6-8 September, Athens, Greece

14. OPEN PEER REVIEW – WHAT IS OPEN?
Definition?
Open Science Fair 2017, 6-8 September, Athens, Greece

15. Anthony Ross-Hellauer (OpenAIRE)
- Open identities: Authors and reviewers are aware of each other’s identity.
- Open reports: Review reports are published alongside the relevant article.
- Open participation: The wider community to able to contribute to the review
process.
- Open interaction: Direct reciprocal discussion between author(s) and reviewers,
and/or between reviewers, is allowed and encouraged.
- Open pre-review manuscripts: Manuscripts are made immediately available (e.g.,
via pre-print servers like arXiv) in advance of any formal peer review procedures.
- Open final-version commenting: Review or commenting on final “version of
record” publications.
- Open platforms: Review is de-coupled from publishing in that it is facilitated by a
different organizational entity than the venue of publication.
Open Science Fair 2017, 6-8 September, Athens, Greece

16. Görögh, Schmidt, Banelyte, Stanciauskas, Woutersen-
Windhouwer: Practices, evaluation and mapping: Methods, tools
and user needs (OpenUP)
• Building upon OpenAIRE’s definition of OPR, the new peer
review methods and tools will be examined in view of main
attributes of openness:
– open identity
– open report
– open participation
– openness in time
– open platform
Open Science Fair 2017, 6-8 September, Athens, Greece

17. Layers of openness in peer review (1)
1.Who opens identity? Author, reviewer, member of the community,
public
2. What is open? Comments, reviews, author's responses, submitted
manuscript, different versions of the manuscript, final version of the
accepted paper
3. Who is reviewing and who is commenting? Community, designated
(formal) reviewers, public
4. What is commented? Manuscript, review, author responses, final
version of the publication, everything
Open Science Fair 2017, 6-8 September, Athens, Greece

18. Layers of openness in peer review (2)
5. When the review takes place? immediately after submission, during
peer-review process, after the final version has been published
6. When is the manuscript/paper available? Immediately after
submission, when the paper is accepted, after the paper was rejected
7. How is the communication process organized? discussions between
reviewers, authors and reviewers, public and reviewers, reviewers and
editors, authors and editors, moderated discussions?
8. Where the process of peer review takes place? journal, separate
platform dedicated for peer review
Open Science Fair 2017, 6-8 September, Athens, Greece

19. WHAT IS OPEN PEER REVIEW?
Disclosed identity of the author(s) 11 9%
Identities of both author and reviewer are disclosed 29 22%
Review is publicly available 55 43%
Disclosed identity of the peer reviewer 26 20%
Author’s responses are publicly available 42 33%
Former versions of the manuscript (before acceptance) are available 22 17%
Post-print peer review 16 12%
Readers can comment submitted and publicly available manuscripts 20 15%
Don’t know 29 22%
Open Science Fair 2017, 6-8 September, Athens, Greece

20. Open Science Fair 2017, 6-8 September, Athens, Greece
14%
10%
8%
10%
17%
12%
35%
26%
22%
33%
29% 29%
7%
18%
29%
Open peer review would enhance peer
review quality
Open peer review would encourage
discussions
Open peer review would endanger
objectivity
1 2 3 4 5

21. OPEN PEER REVIEW AND CROATIAN OA
JOURNALS
• not accepted in Croatian OA journals
• some editors are not familiar with the concept of OPR
• only 7% strongly agree that open peer review would enhance
the peer review quality
Open Science Fair 2017, 6-8 September, Athens, Greece