This webinar, the 3rd and final in the series “Publishing Agricultural Development Research in Social Science Journals”, focuses on the specifics of the referee process—how (and why) to do good reviews, and how to respond to referee comments received. The session includes sample “revise and resubmit” reviews. More info about the series: https://bit.ly/PublishingAgRes

1. WEBINAR # 3
Organized by:
WRITING AND
RESPONDING TO
REFEREE REPORTS
December 7, 2020, 9:00 – 10:30 AM EST
Presenters:
• Cheryl Doss (Oxford / PIM)
• J. V. Meenakshi (Delhi
School of Economics /
SPIA)
Chair:
Michèle Mboo-
Tchouawou
(AWARD)
Webinar series
PUBLISHING AGRICULTURAL
DEVELOPMENT RESEARCH IN
SOCIAL SCIENCE JOURNALS
https://bit.ly/PublishingAgRes

2. A. Writing reviews
Should I agree to
review a paper?
Yes:
if it is from a reputable journal (see session 1)
if it is from a journal you hope to publish in
if it is a paper that you would like to read
even if you are a young scholar (even a PhD student)
No:
if it is completely outside your area of expertise in all
dimensions
if you cannot provide the review in a timely manner
Check in
with editor:
if you have a potential conflict of interest
you would need an extension

3. Really? Do I have to?
• Reviewing papers for journals is part of the service that we all do for
the profession.
• Writing reviews is great experience to think about how papers are put
together and their strengths and weaknesses.
• You should write the kind of review that you would like to receive.
Not necessarily an “accept” but constructive comments to help the
authors produce a better paper.
• If you can’t do the review, responding promptly and suggesting an
alternative reviewer will be appreciated.

4. Reviews may include:
• Responses to specific questions, depending on the journal.
• Comments to the author
• Comments to the editor
• Recommendation to the editor

5. Comments to author: Structure
• Start with the big picture. Write a couple of sentences about what the
paper aspires to do and what it actually does.
• What does it do well? Does it make a contribution to the literature?
• What are your big picture concerns about the paper? They may be
about the framing of the paper, the methods and empirical analysis,
or the relevance of the paper to the journal audience.
• Briefly make broad comments on each section. And then detail any
minor comments.

6. Comments to author: Hints
• If possible, skim the paper when you first receive it, to make sure it is within your
area of expertise (the abstract may not tell you that).
• It may be appropriate to suggest other literature or bodies of literature that the
paper should engage with.
• You should not suggest that they read and incorporate much of your own work.
• You are evaluating the paper that they wrote – it is not helpful to suggest that they
should have written a different paper on that topic. The question is whether this
paper makes a contribution.
• It is useful to provide some concrete suggestions of what they could do to improve
the paper.
• Identify both the aspects of the paper that are successful as well as what needs
improving. That provides encouragement and otherwise that part may get cut in
revisions.
• It takes time to write a good referee report.

7. What not to do as a reviewer
• You are not expected to replicate their model or their results.
• On technical material, you are not responsible for determining
whether it is correct.
• You can note when results seem implausible.
• It is preferable that you not make your recommendation about the
article’s suitability for publication in the text of the comments to the
author.
• Unless you are recommending that it be accepted as it is, you do not
need to correct typos. If there are many typos, you can note that it
needs to be proofread. But you are a reviewer, not a copy editor.

8. Comments to Editor
• The editor will read your comments to the author. You may sum up
the key points, but do not need to provide detail here. You can also
express enthusiasm or scepticism.
• You may have concerns to share with the editor: concerns about
possible plagiarism, too similar to another paper by the same or a
different author, conflict of interest…
• You can also note if there are aspects of the paper that you are
unable to review.

9. Recommendation:
• You will be asked to make a recommendation.
• In the comments to the editor, you can provide more detail on your
reasoning.
• Different journals have different categories –
• Accept, minor revision, major revision, revise and resubmit, reject and
resubmit, reject
• The editor is responsible for making the final decision

10. Decision tree for reviewers
No potential for a
substantial contribution;
methods are not robust;
and/or the data is not
appropriate to answer the
question
Decision
recommendation:
Reject
Some potential to make a
substantial contribution but
this has been not fully or
adequately developed
Additional data analyses
will be needed
Decision
recommendation:
Reject/revise and
resubmit
Substantial reworking or
re-framing of paper will
be required
Decision
recommendation:
Reject/revise and
resubmit
Clear potential for a
substantial contribution to
the literature. Well written
and well executed.
Reworking or re-framing
particular sections is
required
Decision
recommendation:
Major revision
Minor edits and
clarification required
Decision
recommendation:
Minor revision

11. Reviewing revisions
• If authors are asked to revise the paper, you may be sent the paper to review
again.
• Please review it again if at all possible – and quickly.
• Read the editors’ letter and the reports from the other reviewers.
• Read the response document.
• Generally, do not introduce major new concerns about things that were in the
previous version. (Unless they have clarified something that makes it clear
that what they are doing isn’t appropriate.)
• Discuss whether they have adequately responded to your concerns and
whether there are remaining concerns.
• If the paper is likely to be accepted, you may note some stylistic issues in your
report.

12. B. Responding to reviews
Journal
response
Reject
Desk reject (30 to 50%)
Reject after review (55 to
40%)
Revise and
resubmit
Major revision (15-10%)
often multiple rounds
Accept (10-8%)
Reject (5-2%)
Minor revision Accept
Accept

13. Desk reject
Usual reasons:
• Topic not suitable for journal audience
• Editor’s assessment that the paper has weaknesses or does not
constitute a significant contribution to the literature
Before sending this out to another journal:
• Do homework to ensure that the paper is a good fit for the journal. If
necessary rewrite the introduction
• Address reasons raised by editor

14. Reject after review
Take a deep breath and suppress the urge to be defensive!
• Editors are keen to publish good papers. But if none or only a minority of the referees
are/is enthusiastic about your research, need to go back to the drawing board.
Before sending it out to a journal again:
• Go through the same process of selecting a journal. If time to publication is a concern,
factor that in.
• Generally not a good idea to submit the paper in the same form to a lower ranked
journal, and not just because there’s a reasonable probability that it will go back to the
same referee.
• Address substantive comments made; spend the time to re-estimate, re-analyze, re-
interpret, re-orient as necessary and re-write accordingly!
• In other words, treat the rejection as a revise and resubmit, except that a response to
referees and editor is not necessary.

15. Major revise and resubmit
Congratulations, the odds of publication increased substantially!
However:
• It is important to take reviewer’s comments seriously, and not try to either
(a) word-smith a way out of addressing critique (b) provide a rebuttal of
why the referees are wrong. It is easy to spot (a); while (b) is tantamount
to questioning the editor’s judgement in her choice of referees. Neither is
a harbinger of a positive outcome.
• Most referees will distinguish between major and minor comments. If not,
the first step is to parse these out. The editor will also often provide
guidance on what she thinks are the most important comments to take on
board.
• Once you’ve decided to incorporate referees’ suggestions and have
rewritten the paper, it is time to write your response to the referees and
the editor.

16. Writing a referee response
• Major revise and resubmits often go back to the referees.
• Take the time to detail how you have incorporated comments. It is
not uncommon for responses to a single referee to span 5 to 6 pages.
• Copy each of the referee’s comments in the same order as written,
and detail your response immediately below that.
• Make sure that the response refers to page numbers, paragraph
numbers and line numbers so that it is easy for the editor, and the
referees to refer to the revised text and satisfy themselves that their
comments have been taken seriously and incorporated into the
analysis.
• Some specific examples may be instructive…

17. Major r & r, example 1:
The referee suggests a different empirical strategy. You try it and it
leads to different results, explicitly acknowledge it.
Referee A comment: I cannot see how you can determine meaningful Y
estimates without including Z it in the modeling as a covariate…. Bottom line, Z
influences Y which leads to biased results if not included in the model. Please
show the results on Z (descriptive and in the models).
Response: We thank the referee for pointing out the omission of Z as a variable.
We have re-estimated all models after incorporating taste. The Z parameters are
highly significant and positive in all the models (Tables aa and bb). As a
consequence, we have overhauled the Y estimates (including double difference
estimates) in Tables cc to dd, which are now all new. These results vary from the
previous version in the following ways:….

18. Major r & r, example 2:
The referee does not feel that the paper makes a sufficient case of how
it adds to the literature.
Referee A comment: The paper is far too descriptive and superficial. There is
scant detail about what theoretically drives the observed patterns. It needs
better connection between theory-case-discussion.
Response: We appreciate the opportunity to clarify how our analysis draws on
the theoretical literature on feminization of agriculture, and the broader
implications of our study. We have reworked the literature review to highlight
the major questions about the effect of migration on agricultural labor and
decision-making (see p. aa). In the discussion section (pp. bb) we return to
these questions, highlighting what our study shows, and the importance of also
considering key aspects of intersectionality that have received insufficient
attention in other studies.

19. Major r & r, example 3:
If the referee thinks the interpretation is over sold (and you agree),
acknowledge it.
Referee B comment: I still think that the empirical estimates of Y1 and
Y2 are not reliable, and inference based on these is questionable.
Response: As noted on page aa lines bb and cc, we acknowledge the
limitations of estimating a zz function using a single cross-section. We
note also that all our comparisons involving the Y1 and Y2 are based on
95% confidence bands based on multiple outcomes, thus giving us a
measure of confidence in the robustness of the results.

20. Major r & r, example 4:
You disagree with a specific comment of the referee, provide a detailed
explanation.
Referee C comment: I did not like the analysis in section xx (at all). It
does not directly speak to the theoretical framework that is presented,
and interpreting the coefficients is far from straightforward…. I propose
to drop all xx-based results and rewrite the paper as a much shorter
and more focused contribution.
Response: We believe that the analysis in section xx is still of interest,
given the recent work showing that ….. Papers that highlight the
importance of this kind of analysis include aa (2018); bb (2019) and cc
(2019). Their results show that ….. For this reason, we prefer to retain
the analysis in section xx, but have made it more succinct.

21. Major r & r, example 5:
The referee asks for analysis that is not feasible given you data. Say so
explicitly.
Referee D comment: I do not believe that these estimates can be
vested with causal interpretation without panel data because you are
unable to capture plot-level heterogeneity
Response: We agree that use of panel data would greatly enrich this
analysis. To account for some this heterogeneity, we have included soil
characteristics (see page aa). Furthermore, we discuss possible reasons
for bias and why they may not be as much of a concern given the
present analysis (page bb, lines cc to dd). For instance,…

22. Major revise and resubmit:
Provide a summary of changes made to the editor in the cover letter.
Example:
“Thank you for giving us the opportunity to revise this manuscript for aa. We
summarize the main changes below.
• Reviewer a’s comments focused on the estimation strategy, which could cast
doubt on the robustness of our results. In this revision, we have taken this into
account, and we now document this on pages aa and bb. Our main results largely
hold up, though some of the findings are somewhat weaker
• Reviewer b questioned the interpretation of our results in table bb. We provide a
more detailed explanation of our interpretation and draw attention to ….
• We have a difference of opinion with one of the points raised by reviewer c. She
does not agree that the analysis of xx is important, and we respectfully disagree.
We refer to recent work that underscores the importance of this kind of analysis.

23. Concluding thoughts…
Referees are vested in helping authors become better researchers…this is the
main aim of peer review.
Editors are busy: the easier you make it for them, the easier it will be for them
to reach a decision.
Take revise and resubmits seriously. Two main reasons why papers may get
rejected after a revise and resubmit:
• Did not address all comments or response seems casual
• Referees not convinced that their concerns have been addressed
In the end, the paper reflects inputs and judgement of the best experts in the
area and helps it become the best that it can be…

24. Recoding of this webinar will be available on
the PIM website shortly after the live event. All
registrants will receive a follow-up email with
the link to the webinar materials (video,
presentation, podcast).
This webinar series page:
https://bit.ly/PublishingAgRes
Previous PIM Webinars:
http://bit.ly/PIMwebinars
If you want to receive alerts about future
PIM Webinars, sign up here:
https://pim.cgiar.org/subscribe/
@PIM_CGIAR
@PIM.CGIAR
https://bit.ly/PublishingAgRes