1. Lecture 32: Open Peer
Review
 Module 13
 Dr. Jessica Laccetti

2. Outline
 Peer Review
 Open Peer Review
 Your Turn
 Homework

3. Peer Review
 Helps us find the “signal in the noise” (Leigh Dodds)

4. Peer Review
 Experts reviewing your paper/poster/chapter/book
 According to Dr. Vivian Siegel (Research Professor of
Medicine and Cell and Developmental Biology at
Vanderbilt University), journals employ peer review:
 to find out whether the conclusions drawn by the
research are justified and “new” (i.e. whether you have
a right to claim the discovery as “yours”)
 to get some gauge as to the significance or potential
significance of the work to help decide whether your
work is a match for the journal

5. Image from Peer Review in Scientific Publications: http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf

6. Types of Peer Review
 Most common system = “single blind” peer review in which the author’s
name and institution is known to the reviewer, but the reviewer’s name
is not provided to the author.
 A number of journals instead choose to operate a “double blind” peer
review system which is fully anonymised (i.e. the author(s) are unaware
of the identity of the reviewer(s) and vice versa).
 Recently, there have been some experiments with a third type, “open”
peer review, in which the authors’ and reviewers’ names are revealed to
each other. […] Open peer review can be reasonably described as an
experimental system at this stage and is far from common.
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf

7. Draw-Backs of Traditional
Peer Review
 The traditional ways of scientific publishing and
peer review do not live up to the needs of efficient
communication and quality assurance in today’s
highly diverse and rapidly developing world of
science.

8. Advantages of Open Peer
Review
 Open access is fully compatible with traditional
peer review, and beyond that it enables interactive
and transparent forms of review and discussion
open to all interested members of the scientific
com- munity and the public (interactive open
access peer review alias public, collaborative or
community peer review).
 Ref: 294 Liber Quarterly Volume 19 Issue 3/4 2010

9. Advantages of Open Peer
Review
 Open access gives reviewers more information to
work with, i.e., it provides unlimited access to
relevant publications across different scientific
disciplines and communities (interdisciplinary
scientific discussion and quality assurance).
 Ref: 294 Liber Quarterly Volume 19 Issue 3/4 2010

10. Advantages of Open Peer
Review
 Open access facilitates the development and
implementation of new metrics for the impact and
quality of scientific publications (combination of citation,
download/usage, commenting and ranking by various
groups of readers and users, respectively).
 Open access helps to overcome the obsolete
monopoly/oligopoly structures of citation counting
which have been stifling innovation in scientific research,
publishing and evaluation over decades.
 Ref: 294 Liber Quarterly Volume 19 Issue 3/4 2010

11. Advantages of Open Peer
Review
 The possibility of comparing a final revised paper
with the preceding discussion paper and following
the interactive peer review and public discussion
also facilitates the evaluation of individual
publications for non-specialist readers and
evaluators.
 The style and quality of interactive commenting
and argumentation provide insights that go
beyond, and complement, the information
contained in the research article itself.

12. Findings of U.K. Parliament
Report
 We conclude that different types of peer review are
suitable to different disciplines and research
communities.
 We consider that publishers should ensure that the
communities they serve are satisfied with their choice
of peer-review methodology.
 Publishers should keep them updated on new
developments and help them experiment with
different systems they feel may be beneficial.
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf

13. Recommendations of U.K.
Parliament Report
 The principles of openness and transparency in open
peer review are attractive, and it is clear that there is
an increasing range of possibilities.
 There are mixed results in terms of acceptance
amongst researchers and publishers, although some
researchers are keen to see greater transparency in
their fields.
 We encourage publishers to experiment with the
various models of open peer review and transparency
and actively engage researchers in taking part.
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf

16. PLoS
 The approach adopted by PLoS ONE—where the
peer review process focuses solely on whether the
findings and conclusions are justified by the results
and methodology presented, rather than on
assessment of the relative importance of the
research or perceived level of interest it will
generate—has both reduced the burden on the
reviewer and the time it takes to get a paper
published

20. http://mediacommons.futureofthebook.org
/mcpress/ShakespeareQuarterly_NewMe
dia/shakespeare-remediated/murphy-
shakespeare-goes-digital/

22. Your Turn
 In small groups and in a google doc that is public and editable:
 Respond to the aforementioned conclusions found by the U.K.
Parliamentary Report
 Think about what kind of peer review would be best suited to
your own field of study (blind, double-blind, open, closed)
 How might you improve the quality of peer review?
 Post a link to your PUBLIC google doc on the blog
 Pick a link to someone else’s response and add some comments
in the form of an academic peer review

23. Homework
 Online class! By the end of class (9:50) please complete the following exercise:
 As the new editor–in–chief of a significant journal published by Reed Elsevier
you would like to modernize the academic publishing process. You are eager to
implement “open peer review” See
http://en.wikipedia.org/wiki/Open_peer_review even after Nature’s experiment
with this form of peer review failed (see
http://www.nature.com/nature/peerreview/debate/nature05535.html).
However, a more recent experiment by Noah Wardrip-Fruin on the Grand Text
Auto blog was more successful (see http://grandtextauto.org/2009/05/12/blog-
based-peer-review-four-surprises/). Do you try to convince your colleagues to
try open peer review or are you daunted by examples such as Nature’s?
 Upload your 3-5-paragraph response to Google docs. Make sure you share your
document so it is visible to anyone and add a link to the document as a comment
on the Lecture 33 post. Be sure to e-mail the link to me and your TA as well.