Validation, Evaluation, Dissemination: 	
Academia’s gravest problems show
the way to the Next Generation
Repositories
Pand...
Essential scholarly communication processes
controlled exclusively by journals
Validation
Richard Smith, former British Medical Journal editor
https://www.timeshighereducation.com/content/the-peer-revi...
Validation
Does it ensure quality, reproducibility,
reliability?
http://www.cochrane.org/MR000016/METHOD_editorial-peer-re...
http://www.nature.com/nature/journal/v483/n7391/full/483531a.html#t1
Validation
85 per cent of preclinical studies could n...
Validation
http://science.sciencemag.org/content/349/6251/aac4716.full
Does it ensure quality, reproducibility,
reliabilit...
Validation
http://brain.oxfordjournals.org/content/123/9/1964
Does it ensure quality, reproducibility,
reliability?
Validation
http://www.nature.com/news/2011/111005/full/478026a.html
Does it ensure quality, reproducibility,
reliability?
Validation
Pandelis, I have good
news. Pablo says he
will send the Frontiers
paper to …a very
good friend of ours
Does the...
Validation
Does the system promote ethical
conduct?
Validation
https://pdfs.semanticscholar.org/be9a/21d0e93adeb529f7a50d9d50f9d0a6d8f8be.pdf
Does it recognise quality?
Validation
What is the cost in time and money?
Validation
What is the cost in time and money?
https://www.timeshighereducation.com/news/unpaid-peer-review-is-worth-19bn/...
Validation
What we need:!
✓Many reviewers!
✓Experts in the field(s)!
✓Open reviews!
✓Transparent dialogue!
✓Reviewer recogn...
Evaluation
Receive credit based on “where” instead
of “what”
Evaluation
Receive credit based on “where” instead
of “what”
University Rankings
Evaluation
What does the name of the journal say
about its articles?
Numberofcitations
Number of articles
http://e-archivo...
Evaluation
What does the name of the journal say
about its articles?
http://journal.frontiersin.org/article/10.3389/fnhum....
Evaluation
What does the name of the journal say
about its articles?
http://journal.frontiersin.org/article/10.3389/fnhum....
Evaluation
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0056180
What does the name of the journal say
...
Evaluation
The journal long-tailed hierarchy
Evaluation
What we need:!
✓Multi-parametric indexes!
✓Overall assessment!
✓Dynamic assessment
What we have:!
✓Single index...
An alternative model
http://www.openscholar.org.uk/wp-content/uploads/2015/08/nsap_perakakis_2010.pdf
An example repository with
overlay review services
http://sjscience.org
The first Open Peer Review
Module for OA repositories
http://www.openscholar.org.uk/institutional-repositories-start-to-of...
The first Open Peer Review
Module for OA repositories
The first Open Peer Review Module for OA repositories
The first Open Peer Review Module for OA repositories
Building an alternative
reputation system
http://digital.csic.es/bitstream/10261/130842/1/Reputation_academic_world_Sierra...
Validation, Evaluation, Dissemination: 	
Academia’s gravest problems show
the way to the Next Generation
Repositories
Pand...
Nächste SlideShare
Wird geladen in …5
×

Validation, Evaluation, Dissemination: Academia’s gravest problems show the way to the Next Generation Repositories

1.723 Aufrufe

Veröffentlicht am

Talk at the COAR2016 general assembly meeting. Vienna, 12 April 2016

Veröffentlicht in: Wissenschaft
0 Kommentare
3 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
1.723
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
123
Aktionen
Geteilt
0
Downloads
27
Kommentare
0
Gefällt mir
3
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Validation, Evaluation, Dissemination: Academia’s gravest problems show the way to the Next Generation Repositories

  1. 1. Validation, Evaluation, Dissemination: Academia’s gravest problems show the way to the Next Generation Repositories Pandelis Perakakis, PhD ! Open Scholar, CIC University of Granada @os_soc @ppandelis openscholar.org.uk perakakis@openscholar.org.uk COAR annual meeting, Vienna 2016
  2. 2. Essential scholarly communication processes controlled exclusively by journals
  3. 3. Validation Richard Smith, former British Medical Journal editor https://www.timeshighereducation.com/content/the-peer-review-drugs-dont-work May 28, 2015
  4. 4. Validation Does it ensure quality, reproducibility, reliability? http://www.cochrane.org/MR000016/METHOD_editorial-peer-review-for-improving-the- quality-of-reports-of-biomedical-studies
  5. 5. http://www.nature.com/nature/journal/v483/n7391/full/483531a.html#t1 Validation 85 per cent of preclinical studies could not be replicated Does it ensure quality, reproducibility, reliability?
  6. 6. Validation http://science.sciencemag.org/content/349/6251/aac4716.full Does it ensure quality, reproducibility, reliability?
  7. 7. Validation http://brain.oxfordjournals.org/content/123/9/1964 Does it ensure quality, reproducibility, reliability?
  8. 8. Validation http://www.nature.com/news/2011/111005/full/478026a.html Does it ensure quality, reproducibility, reliability?
  9. 9. Validation Pandelis, I have good news. Pablo says he will send the Frontiers paper to …a very good friend of ours Does the system promote ethical conduct?
  10. 10. Validation Does the system promote ethical conduct?
  11. 11. Validation https://pdfs.semanticscholar.org/be9a/21d0e93adeb529f7a50d9d50f9d0a6d8f8be.pdf Does it recognise quality?
  12. 12. Validation What is the cost in time and money?
  13. 13. Validation What is the cost in time and money? https://www.timeshighereducation.com/news/unpaid-peer-review-is-worth-19bn/ 402189.article
  14. 14. Validation What we need:! ✓Many reviewers! ✓Experts in the field(s)! ✓Open reviews! ✓Transparent dialogue! ✓Reviewer recognition What we have:! ✓2-3 reviewers! ✓Anonymous! ✓Non-public reviews! ✓Editorial decision! ✓System of favours
  15. 15. Evaluation Receive credit based on “where” instead of “what”
  16. 16. Evaluation Receive credit based on “where” instead of “what” University Rankings
  17. 17. Evaluation What does the name of the journal say about its articles? Numberofcitations Number of articles http://e-archivo.uc3m.es/bitstream/handle/10016/10968/ we1109.pdf;jsessionid=20173C78597F75F03057340D43D90102?sequence=1
  18. 18. Evaluation What does the name of the journal say about its articles? http://journal.frontiersin.org/article/10.3389/fnhum.2013.00291
  19. 19. Evaluation What does the name of the journal say about its articles? http://journal.frontiersin.org/article/10.3389/fnhum.2013.00291
  20. 20. Evaluation http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0056180 What does the name of the journal say about its articles?
  21. 21. Evaluation The journal long-tailed hierarchy
  22. 22. Evaluation What we need:! ✓Multi-parametric indexes! ✓Overall assessment! ✓Dynamic assessment What we have:! ✓Single index! ✓Mostly impact! ✓Static assessment
  23. 23. An alternative model http://www.openscholar.org.uk/wp-content/uploads/2015/08/nsap_perakakis_2010.pdf
  24. 24. An example repository with overlay review services http://sjscience.org
  25. 25. The first Open Peer Review Module for OA repositories http://www.openscholar.org.uk/institutional-repositories-start-to-offer-peer-review-services/
  26. 26. The first Open Peer Review Module for OA repositories
  27. 27. The first Open Peer Review Module for OA repositories
  28. 28. The first Open Peer Review Module for OA repositories
  29. 29. Building an alternative reputation system http://digital.csic.es/bitstream/10261/130842/1/Reputation_academic_world_Sierra.pdf
  30. 30. Validation, Evaluation, Dissemination: Academia’s gravest problems show the way to the Next Generation Repositories Pandelis Perakakis, PhD ! Open Scholar, CIC University of Granada @os_soc @ppandelis openscholar.org.uk perakakis@openscholar.org.uk COAR annual meeting, Vienna 2016

×