A workshop hosted by the South African Journal of Science aimed at early career researchers with little or no experience in peer reviewing journal articles. A recording of the workshop is available here: https://youtu.be/AGIpuBodZA0

1. Workshop: How to peer
review for a scholarly
journal
5 October 2022
@SAJS_Official @ASSAf_Official
#PeerReview

2. 5 October 2022
How to peer review for a
scholarly journal
@SAJS_Official @ASSAf_Official
#PeerReview
PROGRAMME
14:00 – 14:05 Welcome
14:05 – 14:35 Receiving a peer review invitation – Jemma Finch
14:35 – 15:05 Writing a peer review report – Caradee Wright
15:05 – 15:15 Comfort break
15:15 – 15:25 From peer review report to decision – Floretta Boonzaier
15:25 – 15:40 How to start peer reviewing – Leslie Swartz
15:40 – 16:00 Q&A

3. 5 October 2022
How to peer review for a
scholarly journal
@SAJS_Official @ASSAf_Official
#PeerReview
Caradee Wright
Chief Specialist
Scientist, SAMRC
SAJS Outstanding
Reviewer Awardee
2021
Floretta Boonzaier
Professor of
Psychology, UCT
Associate Editor:
SAJS
Jemma Finch
Senior Lecturer,
UKZN
Associate Editor:
SAJS
Leslie Swartz
Professor of
Psychology, SUN
Editor-in-Chief:
SAJS
Meet the presenters…

4. Receiving a peer
review invitation
Jemma Finch
5 October 2022
How to peer review for a
scholarly journal
@SAJS_Official @ASSAf_Official
#PeerReview

5. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
Outline of topics
• What is peer review?
- Purpose vs perceptions vs ideals
• The peer review process
- From the journal’s perspective
- Peer review models and reform
• Receiving an invitation
• Conflicts of interest
• Benefits of peer review
• Closing thoughts

6. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
What is the Purpose of Peer Review?
“The peer review process assists the scientific community in
assuring the quality of the research before it is published and
before it can be examined and used by a wider audience”
(Cargill and O’Connor, 2013, p. 92)
Cargill, M., O’Connor, P. (2013). Writing scientific research articles: strategy and steps
(2nd edition). Wiley-Blackwell, Oxford, 2013, 223 pp.

7. @SAJS_Official @ASSAf_Official
#PeerReview
Cartoon by
Nick D. Kim
strange-
matter.net
What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts

8. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
What is the Purpose of Peer Review?
• Baldwin (2021: 601) questions the goals of the peer review
process:
“to make sure articles clear a minimum bar of
scientific quality?”
“to select innovative articles?”
“to give authors feedback that helps them
publish their best work?”
Baldwin, M. (2021) To reform peer review, we need to understand its past. Nature
Reviews Physics 3, 600-601 https://doi.org/10.1038/s42254-021-00354-x

9. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
SAJS Guidelines for Reviewers
• AE requires an expert opinion on the quality and suitability
of the manuscript for the SAJS and also to give feedback
to authors that will help them to improve their work
• Please be collegial in your report, both in purpose and
tone
• Identify weaknesses, but also be constructive and have in
mind how the manuscript might be improved for possible
publication
• Identify strong points and valuable research findings
https://sajs.co.za/guidelines-reviewers

10. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
Peer Review Ideals
• Unbiased
• Fair
• Rigorous
• Constructive
• Confidential

11. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
The Peer Review Process
Manuscript
received for
consideration
Manuscript
assessed by
editor
Editor sends
for review
Editor
assesses
reviews to
inform
decision
making
Reject
Revisions
required
Accept
Reject
Adapted from: https://authorservices.wiley.com/Reviewers/journal-reviewers/what-is-peer-review/the-peer-review-process.html
Revise and resubmit
Additional
reviews needed
2/3
reports
Selection,
timeframes

12. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
Peer Review Models
• Single anonymous:
- Reviewer remains anonymous
• Double anonymous:
- Both the author and reviewer names are kept
anonymous
• Open:
- Author and reviewer names are shared

13. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
Peer Review Reform (Baldwin, 2021: 600-601):
• Consensus report:
- “referees consult with the editor to produce a consensus
report on a paper, giving authors a single set of critiques”
• Collaborative review:
- “authors, editors and a set of anonymous reviewers
exchange recommendations and responses”
• Open peer review:
- “publish all submitted articles on their platform and invite
post-publication peer review”
Baldwin, M. (2021). Nature Reviews Physics 3, 600-601 https://doi.org/10.1038/s42254-021-00354-x

14. Should I
accept an
invitation
to review?
The manuscript is
within my field of
expertise
No conflict of
interest
I can manage the
workload within
the allocated
timeframes
What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
I’ve been invited to review, what now?
If YES to all,
then accept

15. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
I’ve been invited to review, what now?
Should I
accept an
invitation
to review?
The manuscript is
within my field of
expertise
No conflict of
interest
I can manage the
workload within
the allocated
timeframes
If NO, decline
with a reason
and try to suggest
other reviewers

16. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
I’ve been invited to review, what now?
If NO, give a
reason and try to
suggest other
reviewers
If YES to all,
then accept
Either way, try to respond timeously
(be cognisant of the author on the other end)

17. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
I’ve been invited to review, what now?
• Keep open lines of communication
• How many reviews should I accept?
• How much time should I spend on a peer review?

18. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
What is a Conflict of Interest?
Conflict
of
Interest
Close friendship /
acquaintance
Recent co-authorship /
collaboration (5 yrs)
Same institution
Competing research
interests
There are
exceptions!
If unsure, chat to
the editor

19. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
Benefits of Peer Review
• Develop critical thinking skills
• Keep up with developments in your field of research
• Enhance your CV and track review metrics:
- e.g., Publons/Web of Science
• Service to the academic community

20. What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
@SAJS_Official @ASSAf_Official
#PeerReview
Closing Thoughts: Are you ready to review?
• Improve representation within the academic community
• We all have to start somewhere!
https://errantscience.com/blog/2016/10/26/imposter-syndrome/

21. @SAJS_Official @ASSAf_Official
#PeerReview
What is peer
review?
Peer review
process
Receiving an
invitation
Conflicts of
interest
Benefits of
peer review
Closing
thoughts
Questions?

22. Writing a peer review
report
Caradee Y Wright
South African Medical Research Council and University of Pretoria
5 October 2022
ASSAf Online Workshop: How to peer review for a scholarly journal

23. You’re agreed to review a manuscript
for a journal, now what?
Dive right in?
No. Formulate a plan.

24. Ask yourself an important question:
What is my role as a reviewer?
• Read the manuscript.
• Consider quality / novelty of the
research.
• Provide constructive feedback.
• Help inform the Associate Editor /
Editor.

25. The review process is multi-stage.
First, read the entire manuscript.

26. HINT!
What to think about on the very first
read?
• What is the story here?
• Am I enjoying reading this
manuscript?
• How many times do I feel agitated?
• How many times do I ask a question
of the manuscript that doesn’t seem
to be answered as I read more?

27. The review process is multi-stage.
First, read the entire manuscript.
Second, read it again but start with the
results, then find their aim / objectives /
research question(s). Read the abstract
and conclusion again. Do they align?
Third, start your report file/where you
plan to draft your review.

28. HINT!
By this stage, you already have a very
good ‘sense/feeling’ about the
manuscript.
This makes writing the review much
easier.
[unless your feeling is below zero!]

29. The structure of your report can either
be free or according to journal
requirements.
In general, this is a good model to
follow:
*** If the journal requires comments
online, always put them into a word
document first in case you lose all your
writing when the power goes out.

31. Title of manuscript
Reviewer’s comments
Date
General
Specific
Give page number and line number with a
specific comment
Can give positive feedback here too
Can also give broad comments e.g., General
comment on discussion – please ensure that
requests made for more information on
questions and results in the results section feed
through into the discussion.
[FYI - General length of a review: ???]

32. HINT!
Do’s and Don’t for a reviewer:
• Don’t use expletives, offensive language.
• Don’t be self-promoting of own references.
• Don’t attack the authors.
• Do give concrete steps for how to improve
the manuscript.
• Don’t disclose identity in your report.
• Don’t be a copy-editor.
• Do imagine you are receiving this review for
one of your manuscripts.

34. Why do I review manuscripts for
journals?
I use it to keep up with the literature in
my fields of expertise.
It’s an obligation for my performance
review at the SAMRC to review journal
manuscripts.
It helps support the scientific
community.
I can help Editors and Associate Editors.
Email: cwright@mrc.ac.za

35. From peer review report to
decision
Professor Floretta Boonzaier
5 October 2022
How to peer review for a
scholarly journal
@SAJS_Official @ASSAf_Official
#PeerReview

36. Journal decision-making process
Source: Cormode (2013)
Cormode, G. (2013). Peer review process and editorial decision-making process at journals. Retrieved from
https://www.editage.com/insights/peer-review-process-and-editorial-decision-making-at-journals?refer=scroll-to-1-article&refer-
type=article

37. Reading the peer-review reports
• Narrative Reports vary in quality and style
• Reviewers for SAJS are also asked about the following:
• Does the manuscript fall within the scope of SAJS?
• Is the manuscript written in a style suitable for a non-specialist and
is it of wider interest than to specialists alone?
• Does the manuscript contain sufficient novel and significant
information to justify publication?
• Do the Title and Abstract clearly and accurately reflect the content
of the manuscript?
• Is the research problem significant and concisely stated?
• Are the methods described comprehensively?
• Is the statistical treatment appropriate?
• Are the interpretations and conclusions justified by the research
results?
• Are the results and discussion confined to relevance to the
objective(s)?

38. Reading the peer-review reports
• Reviewers also rate:
• Overall contribution to the field
• Manuscript on language, grammar and tone
• Overall quality
• Reviewers are also asked about a range of other
technical aspects of the manuscript (e.g. tables, figures
etc.)
• Report form asks for a clear recommendation
• Accept
• Revisions required (minor or major)
• Resubmit for review
• Decline

39. Possible outcomes after review
1.accept without any changes (acceptance): the journal will publish
the paper in its original form
2.accept with minor revisions (acceptance): the journal will publish
the paper and asks the author to make small corrections
3.accept after major revisions (conditional acceptance ): the journal
will publish the paper provided the author/s make the changes
suggested by the reviewers and/or editors; editorial review after
revision
4.revise and resubmit (conditional rejection): the journal is willing
to reconsider the paper in another round of decision making after
the authors make major changes; another round of review
5.reject the paper (outright rejection): the journal will not publish
the paper or reconsider it even if the authors make major
revisions
Adapted from Cormode (2013)

40. Editorial decision emerges from:
• Integrated read of the peer review reports
• Editor’s own reading/assessment of the manuscript
• Return to questions asked upon first submission
• How does it fit with the aims and scope of the journal
and the interests of its audience?

41. How to start peer
reviewing
Leslie Swartz
5 October 2022
How to peer review for a
scholarly journal
@SAJS_Official @ASSAf_Official
#PeerReview

42. A recap,
and some
principles
Peer review is central to the academic enterprise, and
is as important as writing your own articles
There is currently a crisis in peer reviewing – it is hard
for journals to find peer reviewers, and without peer
review journals cannot continue to function
We are all peers
We all have a responsibility to keep the peer review
system going
A rule of thumb: for every time I send an article for
review, I should be prepared to review another article

43. What do I want from a peer reviewer?
• I want someone who is
• Competent
• Constructive
• Fair
• Kind
• Clear
• Focused on making may work better
• Not trying to make me a version of themselves

44. “I am not good enough”
Academics and impostor
syndrome – who do NOT
think that they are
impostors?
The academic hierarchy
(“only professors….”)
What do you really need to
know, and what don’t you
need to know?
It is fine to point out the
limitations of your
knowledge in your review
(the editor may well have
chosen a range of
competencies
You don’t have
to pretend

45. “I’m not
sure I am
the right
person to
review
this”
• Make a list of what you think you can and
cannot do
• How important is what you can do to
helping the author and the journal?
• How much of a barrier is what you can’t do
for your ability to be helpful?
Read the paper quickly, and then:
• Don’t be shy to contact the editor and raise
any concerns – we editors are grateful to
you and want to work with you!
If still in doubt

46. Your role
as a
reviewer is
that of a
‘peer
mentor’
(Way, et
al, 2021)
You are not expected to know everything,
but you can make it clear what you do and
don’t know
Four main
areas:
Do I understand the
methods?
Do I have a good idea of
who the audience is for
this journal? Can I stand
as a ‘representative’ of
that audience?
Do I know enough about
the content to
comment?
Have the authors told
me enough about the
context of the work that
I can make an
assessment?
Way DP, Bierer SB, Cianciolo AT, Gruppen L, Riddle JM,
Mavis B. Fundamentals of Scholarly Peer Review: A
Workshop for Health Professions Educators on Practicing
Scholarly Citizenship. MedEdPORTAL. 2021 Aug 2;17:11174.

47. You are not
alone…
• Get help from others and give help
to others, but ONLY ONCE YOU HAVE
ASKED THE EDITOR IF YOU MAY DO
THIS
• Concerns:
• Confidentiality and respect for
authors
• “Ghost reviewing” and
exploitation of more junior
reviewers, hidden from sight
• In this – remember your rights, those
of the authors, and those of the
journal

48. You are
not
alone…
• Consider some options
• As your supervisor/a more
experienced colleague to
include you in peer reviewing
when they get approached
• Approach a more senior
colleague/supervisor to help you
when you get asked to peer
review
• Develop peer review buddy
systems (peer to peer)

49. Peer review buddy systems (1)
• Link in with “shut up and write” – writing support systems
• Journal clubs also useful
• A good place to start, even before you do peer reviews
yourselves:
• Make a commitment that you and your buddies will “peer
review” reviews you yourselves receive from journals.
• Assess the reviews you receive in terms of competence,
respectfulness, constructiveness, etc; and
• Suggest in your group ways in which the reviews could
have been improved

50. Peer
review
buddy
systems
(2)
First, establish rules
of confidentiality for
your group
Always get
permission from the
editor to use a peer
system
Step 1: Each
(independently)
sketch out areas to
be covered in review
Step 2: Distribute the
tasks and each do
your task
Step 3: Review what
one another have
done
Step 4: Assemble full
review

51. • Write in a supportive tone, but be definitive
• Summarize your understanding of the work
• Explain your overall impressions (recommendation /
ratings)
• Base your overall impressions on actual content
• Indicate strengths and weaknesses, providing specific
examples
• Provide suggestions for improvement
• Be detailed and clear
• Explain any descriptors, such as “insufficient method”
Tips for Writing Comments*
* Dudek NL, Marks MB, Wood TJ, Lee AC. Assessing the quality of supervisor’s
completed clinical evaluation reports. Med Educ 2008; 42:816-22.
NOTE: THIS SLIDE IN ITS ENTIRETY IS COPIED FROM Way DP, Bierer SB, Cianciolo AT, Gruppen L, Riddle JM, Mavis B. Fundamentals of Scholarly Peer Review: A
Workshop for Health Professions Educators on Practicing Scholarly Citizenship. MedEdPORTAL. 2021;17:11174. https://doi.org/10.15766/mep_2374-
8265.11174

52. A PLEA
FROM A
JOURNAL
EDITOR
Please do not undermine the
peer review process while it is
underway, BUT
Do not
undermine
the process
Please do give journals feedback
on peer review experiences,
good and bad
Do give
feedback
Remember: we need to peer
review peer review!
Remember

53. Thank you for your participation
5 October 2022
How to peer review for a
scholarly journal
@SAJS_Official @ASSAf_Official
#PeerReview
Please peer review this
workshop
https://forms.gle/KwTzrfmqC7EsoxUv5