Peer review is evolving as the research landscape changes. While peer review ensures research quality, challenges include reproducibility and bias. Journals are addressing issues like transparency, diversity, and standards compliance through initiatives promoting preprints, registered reports, open data badges, and guidelines on inclusion. Overall, peer review remains important but reforms aim to uphold scholarly ideals while encouraging openness, access, and accountability across the research community.

1. Accept with revisions: The
evolving peer review
landscape
Danielle Padula, Scholastica Community Development

2. Circa 1940: Peer review is the bedrock of quality research…
but not without its challenges
“Off to a Patchy Start: Milestones in Journal Peer Review Research,” article and images by Hilda Bastian

3. Scholastica Perspective: Peer review hub for editors,
authors, and reviewers

4. ❏ The state of peer review
❏ How the industry is addressing key challenges
❏ Future outlook: peer review “with revisions”
What we’ll cover

5. “
More than half of the journals accepted the paper, failing
to notice its fatal flaws. Beyond that headline result, the
data from this sting operation reveal the contours of an
emerging Wild West in academic publishing.
- John Bohannon, Who's Afraid of Peer Review?

6. “

7. The replication crisis

8. Peer review is broken.
...or is it?

9. We need a
diagnosis
Is peer review upholding
scholarly ideals/norms?

10. Researchers want to improve not replace peer review
Publons Global State of Peer
Review report (2018)
◎ 66.8% say peer review is
extremely important
◎ 84.8% say institutions should
better recognize contributions
Sense About Science Peer
Review Survey (2009)
◎ Only 32% say peer review is
best it can be
◎ 79% say peer review should
identify best papers,
determine originality/
importance, spot fraud

11. Key questions for editors, authors, and reviewers
◎ Is research reproducibility possible and encouraged?
◎ Are journals operating transparently/following core
standards?
◎ Is equitable access to opportunities and diversity in
scholarship apparent and encouraged?

12. “
The question shouldn’t be whether the process is perfect.
Rather, as with all scientific processes, does this process
yield a better result than might otherwise be expected
without it.
- Todd Carpenter, Ask The Chefs: Peer Review Quality (2019)

13. Triaging peer
review
Time for some revisions

14. Systemic problems in the research culture to address
“Mertonian” scientific norms/ideals
(CUDOS)
◎ Communality
◎ Universalism
◎ Disinterestedness
◎ Organized skepticism
Scientific perceptions (Anderson,
Martinson, & DeVries, 2007)
◎ Secrecy
◎ Particularism
◎ Self-interestedness
◎ Organized dogmatism

15. “Mertonian” scientific norms/ideals
(CUDOS)
◎ Communality
◎ Universalism
◎ Disinterestedness
◎ Organized skepticism
Scientific perceptions (Anderson,
Martinson, & DeVries, 2007)
◎ Secrecy
◎ Particularism
◎ Self-interestedness
◎ Organized dogmatism

16. Reproducibility
Is research reproducibility possible and encouraged?

17. Registered Reports: Peer review before results
Center for Open Science: https://cos.io/rr/

18. “
Registered Reports eliminates the bias against negative
results in publishing because the results are not known at
the time of review.
- Daniel Simons, Professor at U Illinois, Chief Editor of
Advances in Methods and Practices in Psychological
Science

19. Open science badges to encourage reproducibility

20. Open science badges to encourage reproducibility

21. Transparency and
standards
Are journals operating transparently/following core
standards?

22. TOP Guidelines from the Center for Open Science

23. Duty of publishers, funders, non-profits: Establish journal processes
and standards

24. Duty of authors & institutions: Identify trusted journals
and follow research standards
https://thinkchecksubmit.org

25. Preprints as a model for transparency
pre- and post-publication
◎ Green open access
◎ Preprint + open peer
review
◎ Preprint overlay
publishing model
○ E.g. Discrete Analysis

26. “
We encourage authors to post updated versions on the
arXiv if they discover ways in which their papers can be
improved. If notified about these, we will draw attention
to them on the journal website.
- Discrete Analysis, For Authors page

27. MECA: Manuscript Exchange Common Approach
Transfering manuscripts between
journals/systems:
◎ Easier to submit manuscript to
different journal
◎ Reduce frequency of repeat
reviews

28. Is equitable access to opportunities and
diversity in scholarship apparent and
encouraged?
Diversity and
inclusion

29. Launch of the Coalition for Diversity and Inclusion in
Scholarly Communications (C4DISC)
◎ Statement of Principles
◎ Market research/analysis
◎ Best practices and training
resources
◎ Outreach programs

30. “
The value of the Coalition is being able to share that
information across groups, thereby increasing
dissemination and impact. Forming the Coalition sends
the message [...] that as an industry, we’re committed to
addressing these issues.
- Melanie Dolechek, Executive Director of SSP
interview

31. 1. Focus on making journal articles widely accessible
2. Low or no publication fees
3. Track data around diversity and make an action plan
4. Take steps to support ESL authors
5. Make diversity and inclusion part of the journal mission
Steps towards greater diversity and inclusion
blog.scholasticahq.com/post/ways-academic-journals-can-increase-diversity-peer-review/

32. ❏ The state of peer review
❏ How the industry is
addressing key challenges
❏ Future outlook: peer review
“with revisions”
Where we’ve been and where we’re going:
Monitoring the health of peer review

33. Additional resources for journal editors
https://scholasticahq.com/resources

34. THANKS!
@scholasticahq
support@scholasticahq.com