3. Role of peer review
• Journal peer review is just a step in the scientific
process.
• Popular only since 1960s.
• Online publishing and open access have changed
the publishing landscape.
• Traditional peer review models are single blind
peer review and double blind peer review.
4. Criticism of peer review
• Slow
• Expensive
• Inconsistent
• Biased
• Nepotistic
• Open to abuse
6. Open (non-anonymous) peer review
Randomised Controlled Trial (BMJ 1999; 318: 23 – 27):
- no effect on report quality, recommendation, or time taken to review
- increased likelihood of reviewers declining to review
7. Publishing peer review documents
- In all 4 EMBO publications, including EMBO J, EMBO Reports
-‘Peer Review Process File’ shows all referee reports , author responses and editorial
decision letters
- Referees remain anonymous; opt-out is possible
- 95% of take-up rate; willingness of referees to review unchanged
8.
9. Randomised Controlled Trial (BMJ 2010;341:c5729):
- no effect on report quality or recommendation
- higher refusal rate and reviewers take longer
10. Open peer review – BMC-series
• Medical BMC-series titles and some independent journals
operate open peer review and pre-publication history
visible to all
• Full transparency crucial in medical research
• Full credit to the referees
• Both referees and editors are more accountable for
decisions
• Biologists are more sceptical, but BMC Cancer publishes a
lot of basic science with open peer review
11.
12. Peer review at Biology Direct – key aspects
• The author suggests suitable reviewers from the journal's
Editorial Board
• An article is rejected if three Editorial Board Members do not
agree to review it
• If three EBMs formally agree (after initial skim-reading),
authors can pursue publication (with or without revision),
Editor-in-Chief has final word on publication
• Reviewers’ comments and authors’ responses are published
15. Re-review opt-out – BMC Biology
Authors can opt out of re-review; if the editors judge the revisions sufficient,
the article is published, often accompanied by a critical Commentary.
Discussed in Editorial: http://www.biomedcentral.com/1741-7007/11/18.
22. Innovative peer review models
• Open peer review
• Minimal re-review
• Journal cascades
• Separating interest from soundness
23. Separating interest from soundness
Whatever the model, peer review aims to establish
• whether the research and its description/ interpretation is
sound
• level of interest
The importance of interest levels and extent of advance is
different depending on the editorial goals of the journal
Separating soundness from interest level: BMC series, new
‘Mega journals’
24. Journals publishing all sound science
• BMC Research Notes
• SpringerPlus
• PLoS ONE
• Biology Open (BiO)
• Scientific Reports (NPG)
• PeerJ
• F1000Research
25.
26.
27.
28. Innovative peer review models
• Open peer review
• Minimal re-review
• Journal cascades
• Separating interest from soundness
• Separating peer review from journal
29.
30.
31.
32. Conclusions
Traditional peer review is widely used but also widely
criticised.
Innovative models have been proposed to:
- increase transparency of the peer review process,
- reduce the publication cost and burden on
reviewers,
- increase speed of peer review and publication,
- involve the scientific community in the process.
33. Conclusions
Successful innovations:
- open peer review,
- journal cascades.
New initiatives that need to be tested further:
- peer reviewing manuscripts outside journals,
- post publication peer review.
34. Thank you!
Maria Kowalczuk, PhD
Deputy Biology Editor, BioMed Central
Maria.Kowalczuk@biomedcentral.com
Hinweis der Redaktion
I am part of the team of five Biology and Medical Editors at BioMed Central. Our role is maintaining best practice in peer review across all BioMed Central journals, as well as consistency in advising on all aspect of research and publication ethics. The Biology and Medical Editors are responsible for defining BioMed Central’s editorial policies and work closely with in-house editors, external academic editors and the Committee on Publication Ethics (COPE) to ensure that the journals’ editorial processes are consistent with recognized best practice in scholarly publishing.
This is the picture that comes to mind when thinking about peer review process.
“Science is the ultimate peer review environment, and journal peer review has a limited and particular place and role within it.” Kent Anderson, Scholarly Kitchen.Initially manuscripts were assessed only by journal editors. Peer review became more widespread in 1960s with the increasing specialization of science and a rise in submissions and publications.Publishing landscape has changed with the advent of online publishing and open access. The technological revolution has affected the way referees are selected.Widely used, traditional peer review models are single blind peer review and double blind peer review.
Expectations of a reasonable time to publication vary hugely by field. In much of maths and physics, much scientific comms happens in advance of peer reviewed publication, and timescales of a year or more can be seen as fairly acceptable. In life sciences, peer reviewed publication is key, and delays due to reviewers asking for additional experiments etc are increasingly causing concern.
Also operated by BMJ OPEN2006: Nature’s open peer review trial:Despite enthusiasm for the concept, open peer review was not widely popular, either among authors or by scientists invited to comment.http://www.nature.com/nature/peerreview/debate/nature05535.html
Biology Direct type model - in theoretical areas, the disagreements over interpretation between reviewers and authors are key to the intellectual contribution of the paper, so publishing the debate and the difference of opinion is key.There are also limits on the number of times a particular EBM reviews mss from particular authors.
Again – non-OA journals experiment with peer review models, too. The re-review opt-out is proving very popular – Journal of Cell Biology has recentlytaken on a similar approach.
A cross-publisher initiativeTransfers of articles can also happen inter-publisher via initiatives such as the neuroscience peer review consortium
BMC series introduced this idea of separating Soundness from level of interest, and having an inclusive policy, publishing sound research but highlighting the best. This is the basis of several new megajournals – e.gPloS One, SpringerPlus.
This seems to be the only journal that actually depends on post-publication peer review. A lot of journal allow comments (all BMC ones do), and altmetrics, but this is only in addition to pre-publication peer review.Some journals publish ‘provisional’ versions of articles and then allow public comments and discussion, in parallel with formal peer review. E.g. Atmospheric Chemistry and Physics, PeerJPrePrints, Arxive and other pre-print servers.
Traditional peer review is widely used but also widely criticised.A number of innovative models have been proposed to increase transparency of the peer review process, reduce the publication cost and burden on reviewers, increase speed of peer review and publication, and involve the scientific community in the process.Journals have been successful in implementing open peer review in a number of biomedical journals, and decreasing the number of times a manuscript is peer reviewed by enabling transfer of manuscripts and reviewer reports between journals.New initiatives of peer reviewing manuscripts outside journals and post publication peer review still need to be tried and tested.
Traditional peer review is widely used but also widely criticised.A number of innovative models have been proposed to increase transparency of the peer review process, reduce the publication cost and burden on reviewers, increase speed of peer review and publication, and involve the scientific community in the process.Journals have been successful in implementing open peer review in a number of biomedical journals, and decreasing the number of times a manuscript is peer reviewed by enabling transfer of manuscripts and reviewer reports between journals.New initiatives of peer reviewing manuscripts outside journals and post publication peer review still need to be tried and tested.
