C O P E Washington 9 Nov09
•Als PPT, PDF herunterladen•
1 gefällt mir•667 views
The document discusses the roles and responsibilities of journal editors in ensuring the integrity and quality of published research. It outlines how editors must [1] strive to meet reader and author needs while improving the journal, [2] ensure the quality of published material, and [3] preclude business interests from compromising standards. The document also provides examples of reporting biases and conflicts of interest that editors must guard against, such as selective publication of results and ghost writing. Finally, it discusses steps editors can take like using reporting guidelines and checklists to improve transparency and research quality.
![Published Research Flawed, misleading, deceitful ? John Hoey COPE U.S Seminar 2009 Washington, DC [email_address] www.slideshare.com/hoey](https://image.slidesharecdn.com/copewashington9nov09-091109061815-phpapp01/85/C-O-P-E-Washington-9-Nov09-1-320.jpg)
![[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Outline](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie C O P E Washington 9 Nov09
Ähnlich wie C O P E Washington 9 Nov09 (20)
Mehr von John Hoey
Mehr von John Hoey (6)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
C O P E Washington 9 Nov09
- 1. Published Research Flawed, misleading, deceitful ? John Hoey COPE U.S Seminar 2009 Washington, DC [email_address] www.slideshare.com/hoey
- 3. COPE Code of Conduct General duties and responsibilities of Editors Be responsible for everything published in their journals. • Strive to meet the needs of readers and authors • constantly improve the journal • Ensure the quality of the material they publish • champion freedom of expression • Maintain the integrity of the academic record • Preclude business needs from compromising intellectual standards • always be willing to publish corrections, clarifications, retractions and apologies when needed.
- 6. $$ Conflict of Interest Carelessness Incompetence John Bailar’s Pyramid of Manuscript Problems
- 7. a few examples of problems editors face
- 8. Reporting Bias Kay Dickinson, Reporting and other biases in studies of Neurontin for migraine, psychiatric/bipolar disorders, nociceptive pain, and neuropathic pain . August, 2008 http://dida.library.ucsf.edu/pdf/oxx18r10
- 11. Hypothesis testing or Hypothesis generating? Hypothesis testing Hypothesis generating Prior specification before study begins eg. RCT, observational studies, etc. Finding an interesting result among man possible results eg. Survey, cohort study etc..
- 12. Selective publication - Outcome bias publishing the more interesting (usually positive) result Was there an hypothesis? A plan for analysis and reporting of data? In an RCT, this is the primary outcome
- 13. RCT R Rx A Placebo Outcome
- 14. Study design - RCT R Rx A Placebo Outcome Primary Outcome -Specified in Protocol? or fishing expedition?
- 16. Selective publication - Outcome bias (publishing the more interesting result) 1402 outcomes 31% - 59% incompletely reported (40% not reported at all) Chan, A.-W. et al. CMAJ 2004;171:735-740 48 RCTs funded by national granting agency
- 17. Selective publication - Outcome bias (publishing the more interesting result) Primary Outcome Diabetic control 6 months after the end of intensive multithearpy
- 18. Selective publication - Outcome bias (publishing the more interesting result) Interpretation: Intensive multitherapy for patients with poorly controlled type 2 diabetes is successful in helping patients meet most of the goals set by a national diabetes association. However, 6 months after intensive therapy stopped and patients returned to usual care the benefits had vanished, However, 6 months after intensive therapy stopped and patients returned to usual care the benefits had vanished.
- 19. does any of this matter?
- 20. Neurontin (gabapentin) images from Wikipedia which also has a nice summary of court proceedings and results For minor seizures FDA approved 1994 By 2003 one of Pfizer’s best selling drugs Off-label uses account for 90% of sales Multiple small RCTs Benefit for other disorders e.g. migraine
- 21. P Wessely, C Baumgartner, D Klinger, J Kreczi, N … - Cephalalgia, 1987 A seriously flawed RCT - Accepted for publication by someone Bias Example Publication Final negative primary results not published, only positive preliminary results Selective outcome reporting Outcome reported was not primary or secondary outcome Selective statistical analyses 2 nonrandomized patients assigned to neurotin were include with those randomized Spin Emphasis on “positive” outcomes
- 22. 16 Citations P Wessely , C Baumgartner, D Klinger, J Kreczi, N … - Cephalalgia, 1987 Does it matter? General Principles of Migraine Management: The Changing Role of Prevention E Loder, D Biondi - Headache: The Journal of Head and Face Pain, 2005 - Blackwell Synergy Preventive treatment of migraine - SD Silberstein - Trends in Pharmacological Sciences, 2006 - Elsevier Migraine prevention DW Dodick, SD Silberstein - British Medical Journal, 2007 - pn.bmj.com Neuromodulators for Migraine Prevention R Kaniecki - Headache: The Journal of Head and Face Pain, 2008 - Blackwell Synergy Practice parameter: Evidence-based guidelines for migraine headache (an evidence-based review) Stephen D. Silberstein , MD, FACP, for the US Headache Consortium * Neurology 2000;55:754-762
- 25. a typical editor and their support staff... the editor’s desk
- 26. BMJ Editorial Staff Editor in chief Fiona Godlee email T: + 44(0)20 7383 6102 Deputy editors Jane Smith email T:+44 (0)20 7383 6009 Tony Delamothe email T: +44 (0) 20 7383 6006 Trish Groves email T:+44 (0)20 7383 6018 Magazine editor Trevor Jackson email T:+44 (0)20 7383 6677 bmj.com editor David Payne email T:+44 (0)20 7383 6532 Editorials editor Giselle Jones Research papers editors Kristina Fister email Trish Groves email Elizabeth Loder email Christopher Martyn email Alison Tonks email Primary care editor Domhnall MacAuley email News editors Annabel Ferriman email T: 44 (0)20 7383 6035 Zosia Kmietowicz email Features editors Deborah Cohen email T: +44 (0)20 7383 6183 Rebecca Coombes email T: +44 (0)20 7383 6243 Clinical reviews editor Kirsten Patrick email Practice editor Mabel Chew email Analysis editor Tessa Richards email T: +44 (0) 20 7383 61 Letters and obituaries editor Sharon Davies email T:+44 (0)20 7383 6716 Observations and reviews editor Trevor Jackson email T:+44 (0)20 7383 6677 Roger Robinson editorial registrar Helen Macdonald email T: + 44 (0)20 7 874 7022 Senior researcher Sara Schroter email T: +44 (0)20 7383 6744 Patient editor Peter Lapsley email Web team Editor bmj.com David Payne email Assistant editor, bmj.com Birte Twisselmann email T: +44 (0)20 7383 6720 Print journal team Magazine editor Trevor Jackson email Designer Jane Walker email Senior art worker Adam di Chiara email Picture editor Vanessa Fletcher email Deputy managing editor Lucy Banham email T:+44 (0)20 7383 6365 Technical editors Jackie Annis email T:+44 (0)20 7383 6658 Maggie Butler email T:+44 (0)20 7383 6074 Sally Carter email T:+44 (0)20 7383 6659 Margaret Cooter email T:+44 (0)20 7383 6657 Greg Cotton email T:+44 (0)20 7383 6685 Clare Griffith email T: +44 (0)20 7383 6051 Richard Hurley email T: + 44 (0)20 7383 6051 Elizabeth Payne email T: + 44 (0)20 7383 6449 Karl Sharrock email T:+44 (0)20 7383 6658 Barbara Squire email T:+44 (0)20 7383 6658 Julia Thompson email T:+44 (0)20 7383 6691 Douglas Kamerow email PA to editor in chief Julia Burrell email T:+44 (0)20 7383 6102 Departmental administrator Chelsey White email T:+44 (0)20 7383 6109 Benchpress database manager Gary Bryan email T:+44 (0)20 7383 6304 Benchpress administrator Sue Minns email T:+44 (0)20 7383 6309 Chief production editor John Mayor email T:+44 (0)20 7383 6355 Assistant production editor Edwyn Mayhew email T:+44 (0)20 7383 6145 Production editor Jett Aislabie email T:+44 (0)20 7874 7014 Malcolm Brown (maternity cover) T:+44 (0)20 7874 7014 Illustrator Anthea Wilkie email T:+44 (0)1737 215143 Copyright administrator email Career Focus Edward Davies email T: + 44 (0) 20 7383 6562 studentBMJ Jessie Colquhoun [email_address] T: +44 (0)20 7874 7016 Visiting editors Jennifer Leaning (USA) Ray Moynihan (USA) Joanne Roberts (USA) Charlie Wilson (USA) Editorial advisers Steven Reid Ian Maconochie Peter Leman Nick Dunn Frank Sullivan Pippa Oakeshott Aziz Sheikh Lucy Chappell Christopher Whitty Scott Murray Julia Hippisley-Cox Josip Car Robin Fox Sue Morgan Statistical advisers Doug Altman Tim Cole Hazel Inskip Julie Morris Deborah Ashby Jon Deek
- 27. The Lancet Editorial office “ The Apartment” Billy Wilder, Jack Lemnon, 1959
- 28. Editor David Bevan MB Toronto
- 29. Photo by Waldo www.flickr.com/photos/waldo4/2178788631/ Manuscripts are messy & confusing
- 30. Editorial checklists for authors
- 31. RCTs Consort guideline Checklists for KEY elements of a study that need to be reported in published papers. Minimal required content www.equator-network.org
- 32. ? CONSORT Enrolment Allocation Follow-up Analysis
- 35. What can editor’s do to improve quality of what they publish?
- 37. Conflict of Interest -
- 38. Conflict of Interest -
- 42. Thank-you www.equator-network.org [email_address] www.slideshare.com/hoey
Hinweis der Redaktion
- These meanings reflect current popular usage and I’ll use them here. Dickison makes these distinctions.
- The bias is almost always in favour of the positive outcome. There is a growing literature documenting the effect size of these biases. The effect sizes are substantial. Dickinson has a nice bibliography for those interested in a particular bias.
- Chan et al looked at studies funded by the Canadian Institutes of Health Research during the late 1990’s and then tried to find published versions of these studies. They compared the funded protocol with the published papers. Despite careful literature searches and contacts with the funded researchers they were unable to find publications for 40% of the funded trials. (unreported). Of the reported trials between 31% (for efficacy outcomes) and 59% (for harm outcomes) were incompletely reported.
- In a study with several outcome measures - key results, which ones get published? This is clearest for RCTs. An RCT must specify a primary hypothesis before the study begins. This is called the primary outcome and is the main outcome that must be reported. Sample size is usually calculated on the basis of this outcome. Observational studies may also have prespecified hypotheses. These are clearly described in study protocols. Published reports must report the results related to the primary hypthoses. To do otherwise - say to look at all the results and associations and only present those that are positive is misleading to readers: An observational study then becomes a data mining expedition - or fishing expedition. (That’s ok, but it ought to be reported as such - as an exploratory study. Let me give a personal expample. Tell me if it is malfeacance or incompetence? This is a published study in CMAJ. It was an RCT of an intensive intervention for patient with adult onset diabetes mellitus to determine if the intervention led a better outcome for the patients. This is an important question. Patients with diabetes are very difficult to manage, frequently do not follow advice and the disease carries a high risk of serious and fatal events if not properly managed. Thus there is great interest in seeing whether we can improve this situation. Would more intensive management - more frequent tests, diet counselling, nurse specialists encouraging and monitoring patients and so on improve outcomes? The protocol specified several outcomes that were of interest. Indeed a lot of outcomes - the differences between the patients in the control and experimental groups at the end of the study - were available to the authors. In the manuscript submitted, the Abstract (and the discussion of the paper) omitted the sentence that is unreadable on this slide. Thus there conclusion was that intensive therapy was “successful” blah blah blah. The result was positive. But this was not the key result of interest specified as the primary objective of the study. The primary objective was to compare a specific objective after 6 months of therapy had stopped. For that result there was no benefit and this was a negative trial according to our definitions. This was a publicly funded trial. So lets look at RCTs. Do published RCTs report the primary prespecified outcome(s)? One of the more interesting studies of this is work by AnWen Chen and Doug Altman.
- In a study with several outcome measures - key results, which ones get published? This is clearest for RCTs. An RCT must specify a primary hypothesis before the study begins. This is called the primary outcome and is the main outcome that must be reported. Sample size is usually calculated on the basis of this outcome. Observational studies may also have prespecified hypotheses. These are clearly described in study protocols. Published reports must report the results related to the primary hypthoses. To do otherwise - say to look at all the results and associations and only present those that are positive is misleading to readers: An observational study then becomes a data mining expedition - or fishing expedition. (That’s ok, but it ought to be reported as such - as an exploratory study. Let me give a personal expample. Tell me if it is malfeacance or incompetence? This is a published study in CMAJ. It was an RCT of an intensive intervention for patient with adult onset diabetes mellitus to determine if the intervention led a better outcome for the patients. This is an important question. Patients with diabetes are very difficult to manage, frequently do not follow advice and the disease carries a high risk of serious and fatal events if not properly managed. Thus there is great interest in seeing whether we can improve this situation. Would more intensive management - more frequent tests, diet counselling, nurse specialists encouraging and monitoring patients and so on improve outcomes? The protocol specified several outcomes that were of interest. Indeed a lot of outcomes - the differences between the patients in the control and experimental groups at the end of the study - were available to the authors. In the manuscript submitted, the Abstract (and the discussion of the paper) omitted the sentence that is unreadable on this slide. Thus there conclusion was that intensive therapy was “successful” blah blah blah. The result was positive. But this was not the key result of interest specified as the primary objective of the study. The primary objective was to compare a specific objective after 6 months of therapy had stopped. For that result there was no benefit and this was a negative trial according to our definitions. This was a publicly funded trial. So lets look at RCTs. Do published RCTs report the primary prespecified outcome(s)? One of the more interesting studies of this is work by AnWen Chen and Doug Altman.
- Approved by US FDA 1994 RX partial seizures By 2003, one of Pfizer’s best selling drugs for minor seizures. Off-label uses account for 90% of sales migraine, bipolar disorders, OCD, depression, insommnia, etc.. Adverse effects - dizziness, mood swings etc.. hepatotoxcity, depression, suicide, Court cases - for illegally marketing a drug based on no evidence of efficacy
- Dickinson’s analysis of another paper that was used by Pfizer to promote off-label uses of Neurontin. Clearly this is a seriously flawed RCT, as published by an editor, I hope not one in this room.
- Does it matter. Well yes. This misleadingly reported study is still being cited, the drug is still being recommended for prevention of migraine and worse, has found it’s way into guidelines that are Evidence Based, whatever that means in this context.
- Group 2 are highly recommended based on RCT evidence.