Evidence Based Medicine

1. Randomised Controlled Trials
Ninian Peckitt
FRCS FFD RCS FDS RCS FACCS
Oral and Maxillofacial Surgeon / Facial Plastic Surgeon
Adjunct Associate Professor of Engineering Assisted Surgery
Massey University New Zealand

2. Classification of Randomised Trials
• Parallel Group
– Randomly assigned to 2 groups
– Treatment vs non treatment
• Crossover
– Over time random sequence of intervention
• Split Body
– eg Right vs Left Random treatment vs No Treatment
• Cluster
– Pre-existing Groups eg Villages/Schools
• Factorial
– Combination of interventions / non interventions in Random groups

3. Parallel Randomised Trials

4. Classification of Randomised Trials
By outcome of interest (efficacy vs. effectiveness)
• RCTs - "explanatory"
– Test efficacy
– Highly selected participants
– Highly Controlled conditions
• RCTs – “pragmatic.”
– Test effectiveness in every day practice
– Unselected Participants
– Flexible Conditions
– Give informed decisions about practice

5. Classification of Randomised Trials
By hypothesis (superiority vs. noninferiority vs. equivalence)
• "superiority trials" (most)
– statistically significant
• "noninferiority trials”
– new treatment no worse than existing Rx
• "equivalence trials"
- x2 Rx indistiguishable

6. Randomisation
Advantages of proper randomisation
• choosing a randomization procedure - generate unpredictable sequence
• eg simple random assignment to any of the groups at equal probabilities,
– may be "restricted,"
– or may be "adaptive."
• allocation concealment,
– precautions to ensure group assignment not revealed prior to allocating to groups.
– Non-random "systematic" methods of group assignment, such as alternating subjects
between one group and the other, can cause "limitless contamination possibilities" and can
cause a breach of allocation concealment

7. Randomisation - Goals
• Equal Group Sizes for Adequate Statistical Power
(Especially Subgroup Analyses)
• Low selection bias
(investigator cannot predict the next subject's group
assignment by examining which group has been
assigned the fewest subjects up to that point)
• Low probability of confounding
(i.e., a low probability of "accidental bias"),
(i.e. a balance in covariates across groups).

8. Simple Randomisation
• commonly used
• intuitive procedure
• similar to "repeated fair coin-tossing
• AKA "complete" or "unrestricted" randomization
• robust against selection and accidental biases
• possibility of imbalanced group sizes in small RCTs
• recommended only for RCTs > 200 subjects

9. Restricted Randomisation
To balance group sizes in smaller RCTs
• Permuted-block randomization or blocked randomization:
– a "block size" and "allocation ratio" (number of subjects in one group versus the other group) are specified,
– and subjects are allocated randomly within each block.
– For example, a block size of 6 and an allocation ratio of 2:1 would lead to random assignment of 4 subjects
to one group and 2 to the other.
–
• This type of randomization can be combined with "stratified randomization", for example by
center in a multicenter trial, to "ensure good balance of participant characteristics in each group."
[2]
• A special case of permuted-block randomization is random allocation,
– in which the entire sample is treated as one block.[29]
• The major disadvantage of permuted-block randomization is that even if the block sizes are large
and randomly varied, the procedure can lead to selection bias.[30]
• Another disadvantage is that "proper" analysis of data from permuted-block-randomized RCTs
requires stratification by blocks.[32]

10. Restricted Randomisation
• Adaptive biased-coin randomization methods
• urn randomization is the most widely-known type):
• In these relatively uncommon methods, the probability of being
assigned to a group decreases if the group is over-represented and
increases if the group is under-represented.[29]
• The methods are thought to be less affected by selection bias than
permuted-block randomization.[32]

11. Adaptive Randomisation
• Covariate-adaptive randomization,
• minimization:
– The probability of being assigned to a group varies in order to minimize
"covariate imbalance."[32]
– Minimization is reported to have "supporters and detractors";[29]
because only the first subject's group assignment is truly chosen at
random
– the method does not necessarily eliminate bias on unknown factors.[2]

12. Adaptive Randomisation
Response-adaptive randomization, (outcome-adaptive):
• The probability of being assigned to a group increases if the
responses of the prior patients in the group were favorable.[32]
• Although arguments have been made that this approach is more
ethical than other types of randomization when the probability that a
treatment is effective or ineffective increases during the course of an
RCT, ethicists have not yet studied the approach in detail.[33]

13. Allocation Concealment
"Allocation concealment“
(protecting the randomisation process
so that the treatment to be allocated is not known before the patient is entered into the study")
• desirable in RCTs.[34]
• In pratice difficult to maintain impartiality.
• Stories abound of investigators holding up sealed envelopes to lights or ransacking offices to determine group assignments in order to
dictate the assignment of their next patient.[29] Such practices introduce selection bias and confounders (both of which should be
minimized by randomization), thereby possibly distorting the results of the study.[29]
• Adequate allocation concealment should defeat patients and investigators from discovering treatment allocation once a study is underway
and after the study has concluded. Treatment related side-effects or adverse events may be specific enough to reveal allocation to
investigators or patients thereby introducing bias or influencing any subjective parameters collected by investigators or requested from
subjects.
• Some standard methods of ensuring allocation concealment include sequentially-numbered, opaque, sealed envelopes (SNOSE);
sequentially-numbered containers; pharmacy controlled randomization; and central randomization.[29]
• It is recommended that allocation concealment methods be included in an RCT's protocol, and that the allocation concealment methods
should be reported in detail in a publication of an RCT's results; however, 2005 study determined that most RCTs have unclear allocation
concealment in their protocols, in their publications, or both.[35] On the other hand, a 2008 study of 146 meta-analyses concluded that the
results of RCTs with inadequate or unclear allocation concealment tended to be biased toward beneficial effects only if the RCTs'
outcomes were subjective as opposed to objective.[36]

14. Blinding (Masking)
• Prevent from knowing which intervention was received."[36]
• Unlike allocation concealment, blinding is sometimes inappropriate or impossible to perform in an RCT; for
example, if an RCT involves a treatment in which active participation of the patient is necessary (e.g.,
physical therapy), participants cannot be blinded to the intervention.
• Traditionally, blinded RCTs have been classified as "single-blind," "double-blind," or "triple-blind"; however, in
2001 and 2006 two studies showed that these terms have different meanings for different people.[37][38]
• The 2010 CONSORT Statement specifies that authors and editors should not use the terms "single-blind,"
"double-blind," and "triple-blind"; instead, reports of blinded RCT should discuss "If done, who was blinded after
assignment to interventions (for example, participants, care providers, those assessing outcomes) and how."[2]
• RCTs without blinding are referred to as "unblinded",[39] "open",[40] or (if the intervention is a medication) "
open-label".[41]
• In 2008 a study concluded that the results of unblinded RCTs tended to be biased toward beneficial effects only if
the RCTs' outcomes were subjective as opposed to objective;[36]
(e.g. in an RCT of treatments for multiple sclerosis, unblinded neurologists (but not blinded neurologists) felt that
the treatments were beneficial.[42] In pragmatic RCTs, although the participants and providers are often
unblinded, it is "still desirable and often possible to blind the assessor or obtain an objective source of data for
evaluation of outcomes."[27]

15. Analysis of Data from RCTs
• For dichotomous (binary – good/bad) outcome data
– logistic regression (e.g., to predict sustained virological response after
receipt of peginterferon alfa-2a for hepatitis C[43]) and other methods
can be used.
• Continuous data, analysis of covariance (e.g., for changes in blood
lipid levels after receipt of atorvastatin after
acute coronary syndrome[44]) tests the effects of predictor variables
• For time-to-event outcome data that may be censored,
survival analysis (e.g., Kaplan–Meier estimators and
Cox proportional hazards models for time to coronary heart disease
after receipt of hormone replacement therapy in menopause[45]) is
appropriate.

16. Analysis of RCT Data
• Whether a RCT should be stopped early due to interim results. For
example, RCTs may be stopped early if an intervention produces "larger
than expected benefit or harm," or if "investigators find evidence of no
important difference between experimental and control interventions."[2]
• The extent to which the groups can be analyzed exactly as they existed
upon randomization (i.e., whether a so-called "intention-to-treat analysis" is
used). A "pure" intention-to-treat analysis is "possible only when complete
outcome data are available" for all randomized subjects;[46] when some
outcome data are missing, options include analyzing only cases with known
outcomes and using imputed data.[2] Nevertheless, the more that analyses
can include all participants in the groups to which they were randomized, the
less bias that an RCT will be subject to.[2]
• Whether subgroup analysis should be performed. These are "often
discouraged" because multiple comparisons may produce false positive
findings that cannot be confirmed by other studies.[2]

17. Reporting of RCT Results
The CONSORT 2010 Statement
"an evidence-based, minimum set of recommendations for reporting RCTs."[47]
The CONSORT 2010 checklist
– 25 items (many with sub-items)
focusing on "individually randomised, two group, parallel trials"
(which are the most common type of RCT) [1]
For other RCT study designs, "CONSORT extensions" are published.[1]

18. Advantages RCTs
Advantages
• RCTs - the most reliable form of scientific evidence in the
hierarchy of evidence that influences healthcare policy and practice
because RCTs reduce spurious causality and bias. Results of RCTs
may be combined in systematic reviews which are increasingly
being used in the conduct of evidence-based medicine.

19. Hierarchy of Evidence
Greenhalgh suggests that:
"The hierarchy of evidence Standard notation for the relative weight carried by the different types of primary study when making decisions
about clinical interventions (the “hierarchy of evidence”) puts them in the following order*:
1. Systematic reviews and meta-analyses
2. RCTs with definitive results
(confidence intervals that do not overlap the threshold clinically significant effect)
3. RCTs with non-definitive results
(a point estimate suggests a clinically significant effect but confidence intervals overlap threshold for this effect)
4. Cohort studies
5. Case-control studies
6. Cross sectional surveys
7. Case reports."
(Greenhalgh, T. 1997. p244.) ref* to Guyatt et al 1995.

20. Disadvantages - RCTs
Limitations of external validity
• Where the RCT was performed
(e.g., what works in one country may not work in another)
• Characteristics of the patients
(e.g., include better prognosis / exclude "women, children, the elderly, and common medical conditions"[62])
• Study procedures
(pts may receive Rx/care difficult to achieve in the "real world")
• Outcome measures
(e.g., RCTs may use composite measures infrequently used in clinical practice)
• Incomplete reporting of adverse effects of interventions
• Cost

21. Relative Importance of RCTs
• Two studies published in The New England Journal of Medicine in 2000 found that
observational studies and RCTs overall produced similar results.[64][65]
• Doubt?
– "observational studies should not be used for defining evidence-based medical care“
– RCTs' results are "evidence of the highest grade."[64][65]
• Journal of the American Medical Association 2001
– "discrepancies beyond chance do occur
– differences in estimated magnitude of treatment effect are very common"
– observational studies vs RCTs.[66]
• If study designs ranked by their potential for new discoveries, then hierarchy becomes:
1. anecdotal evidence
2. observational studies
3. followed by RCTs.[67]
• RCTs may be unnecessary for treatments that have dramatic and rapid effects relative to
the expected stable or progressively worse natural course of the condition treated.[59]
[68]

22. RCT Difficulties
Difficulty in studying rare events
• Infrequent events /uncommon outcomes
– RCTs with extremely large sample sizes
– may therefore best be assessed by observational studies.[59]
Difficulty in studying outcomes in distant future
• It is costly to maintain RCTs for the years or decades that would
be ideal for evaluating some interventions.[59][60]

23. RCTs – Industrial Bias?
Pro-industry findings in industry-funded RCTs
• A systematic review 2003 - four 1986-2002 articles
• industry-sponsored vs nonindustry-sponsored RCTs
• correlation - industry sponsorship / positive study outcome all studies [70]
• Study 2004 (1999-2001 RCTs )
– leading medical and surgical journals industry-funded RCTs
– "more likely statistically significant pro-industry findings."[71]
• One possible reason for the pro-industry results in industry-funded
published RCTs is publication bias.[71]

24. Therapeutic Misconception
• informed consent usual
• since 1982 RCT subjects believe that they are certain to
receive treatment that is best for them personally
• that is, they do not understand the difference between
research and treatment.[72][73]
• Further research is necessary to determine the
prevalence of and ways to address this "
therapeutic misconception".[73]

25. Statistical Error
RCT Errors
• type I ("false positive")
– typical RCT will use 0.05 (i.e., 1 in 20) as the probability that the RCT
will falsely find two equally effective treatments significantly different.[74]
• type II ("false negative")
– Paper 1978 - sample sizes of many "negative" RCTs
– too small to make definitive conclusions about the negative results,[75]
– by 2005-2006 a sizeable proportion of RCTs still had inaccurate or
incompletely-reported sample size calculations.[76]

26. Cultural Effects
• RCT method creates cultural effects
• Poorly understood[77]
• Terminal pts may attempt to join trials
– as a last ditch attempt at treatment,
– even when treatments are unlikely to be successful

27. References
References
1. ^ a b c d Schulz KF, Altman DG, Moher D; for the CONSORT Group (2010). "CONSORT 2010 Statement: updated guidelines for reporting parallel group
randomised trials". Br Med J 340: c332. doi:10.1136/bmj.c332. PMC 2844940. PMID 20332509.
2. ^ a b c d e f g h i Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG (2010). "CONSORT 2010
explanation and elaboration: updated guidelines for reporting parallel group randomised trials". Br Med J 340: c869. doi:10.1136/bmj.c869. PMC 2844943. PMID
20332511.
3. ^ Ranjith G (2005). "Interferon-α-induced depression: when a randomized trial is not a randomized controlled trial". Psychother Psychosom 74 (6): 387. doi:
10.1159/000087787. PMID 16244516.
4. ^ Chalmers TC, Smith H Jr, Blackburn B, Silverman B, Schroeder B, Reitman D, Ambroz A (1981). "A method for assessing the quality of a randomized control
trial". Control Clin Trials 2 (1): 31–49. doi:10.1016/0197-2456(81)90056-8. PMID 7261638.
5. ^ Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, Mantel N, McPherson K, Peto J, Smith PG (1976). "Design and analysis of randomized clinical
trials requiring prolonged observation of each patient. I. Introduction and design". Br J Cancer 34 (6): 585–612. doi:10.1038/bjc.1976.220. PMC 2025229/. PMID
795448.
6. ^ Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, Mantel N, McPherson K, Peto J, Smith PG (1977). "Design and analysis of randomized clinical
trials requiring prolonged observation of each patient. II. Analysis and examples". Br J Cancer 35 (1): 1–39. doi:10.1038/bjc.1977.1. PMC 2025310. PMID 831755.
7. ^ Wollert KC, Meyer GP, Lotz J, Ringes-Lichtenberg S, Lippolt P, Breidenbach C, Fichtner S, Korte T, Hornig B, Messinger D, Arseniev L, Hertenstein B, Ganser A,
Drexler H (2004). "Intracoronary autologous bone-marrow cell transfer after myocardial infarction: the BOOST randomised controlled clinical trial". Lancet 364
(9429): 141–8. doi:10.1016/S0140-6736(04)16626-9. PMID 15246726.
8. ^ Charles Sanders Peirce and Joseph Jastrow (1885). "On Small Differences in Sensation". Memoirs of the National Academy of Sciences 3: 73–83.
http://psychclassics.yorku.ca/Peirce/small-diffs.htm. http://psychclassics.yorku.ca/Peirce/small-diffs.htm
9. ^ Hacking, Ian (September 1988). "Telepathy: Origins of Randomization in Experimental Design". Isis 79 (3): 427–451. doi:10.1086/354775. JSTOR 234674. MR
1013489. http://www.jstor.org/stable/234674.
10. ^ Stephen M. Stigler (November 1992). "A Historical View of Statistical Concepts in Psychology and Educational Research". American Journal of Education 101 (1):
60–70. doi:10.1086/444032.
11. ^ Trudy Dehue (December 1997). "Deception, Efficiency, and Random Groups: Psychology and the Gradual Origination of the Random Group Design". Isis 88 (4):
653–673. doi:10.1086/383850.
12. ^ Neyman, Jerzy. 1923 [1990]. “On the Application of Probability Theory to AgriculturalExperiments. Essay on Principles. Section 9.” Statistical Science 5 (4): 465–
472. Trans. Dorota M. Dabrowska and Terence P. Speed.
13. ^ According to Conniffe (1991, p. 87), Ronald A. Fisher was "interested in application and in the popularization of statistical methods and his early book
Statistical Methods for Research Workers, published in 1925, went through many editions and motivated and influenced the practical use of statistics in many fields
of study. His Design of Experiments (1935) [promoted] statistical technique and application. In that book he emphasized examples and how to design experiments
systematically from a statistical point of view. The mathematical justification of the methods described was not stressed and, indeed, proofs were often barely
sketched or omitted altogether ..., a fact which led H. B. Mann to fill the gaps with a rigorous mathematical treatment in his well known treatise, Mann (1949)."
14. Page 87: Conniffe, Denis (1990–1991). "R. A. Fisher and the development of statistics—a view in his centenary year". Journal of the Statistical and Social Inquiry
Society of Ireland (Dublin: Statistical and Social Inquiry Society of Ireland) XXVI (3): pp. 55–108. ISSN 00814776. http://www.tara.tcd.ie/jspui/handle/2262/2764.

28. References
15. ^ Brown D (1998-11-02). "Landmark study made research resistant to bias". Washington Post.
16. ^ Shikata S, Nakayama T, Noguchi Y, Taji Y, Yamagishi H (2006). "Comparison of effects in randomized controlled trials with observational studies in
digestive surgery". Ann Surg 244 (5): 668–76. doi:10.1097/01.sla.0000225356.04304.bc. PMC 1856609. PMID 17060757.
17. ^ a b Stolberg HO, Norman G, Trop I (2004). "Randomized controlled trials". Am J Roentgenol 183 (6): 1539–44. PMID 15547188.
http://www.ajronline.org/cgi/content/full/183/6/1539.
18. ^ Meldrum ML (2000). "A brief history of the randomized controlled trial. From oranges and lemons to the gold standard". Hematol Oncol Clin North Am
14 (4): 745–60, vii. doi:10.1016/S0889-8588(05)70309-9. PMID 10949771.
19. ^ Freedman B (1987). "Equipoise and the ethics of clinical research". N Engl J Med 317 (3): 141–5. doi:10.1056/NEJM198707163170304. PMID
3600702.
20. ^ Gifford F (1995). "Community-equipoise and the ethics of randomized clinical trials". Bioethics 9 (2): 127–48. doi:
10.1111/j.1467-8519.1995.tb00306.x. PMID 11653056.
21. ^ Edwards SJL, Lilford RJ, Hewison J (1998).
"The ethics of randomised controlled trials from the perspectives of patients, the public, and healthcare professionals". Br Med J 317 (7167): 1209–12.
PMC 1114158. PMID 9794861. http://www.bmj.com/cgi/content/full/317/7167/1209.
22. ^ Zelen M (1979). "A new design for randomized clinical trials". N Engl J Med 300 (22): 1242–5. doi:10.1056/NEJM197905313002203. PMID 431682.
http://content.nejm.org/cgi/content/abstract/300/22/1242.
23. ^ Torgerson DJ, Roland M (1998). "What is Zelen's design?". Br Med J 316 (7131): 606. PMC 1112637. PMID 9518917.
http://www.bmj.com/cgi/content/full/316/7131/606.
24. ^ a b Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG (2010).
"The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed". BMJ 340: c723. doi:10.1136/bmj.c723.
PMC 2844941. PMID 20332510. http://www.bmj.com/cgi/content/full/340/mar23_1/c723.
25. ^ Jones, Byron; Kenward, Michael G. (2003). Design and Analysis of Cross-Over Trials (Second ed.). London: Chapman and Hall.
26. ^ Vonesh, Edward F.; Chinchilli, Vernon G. (1997). "Crossover Experiments". Linear and Nonlinear Models for the Analysis of Repeated Measurements
. London: Chapman and Hall. pp. 111–202.
27. ^ a b c d Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D; CONSORT group; Pragmatic Trials in
Healthcare (Practihc) group (2008). "Improving the reporting of pragmatic trials: an extension of the CONSORT statement". BMJ 337: a2390. doi:
10.1136/bmj.a2390. PMID 19001484. http://www.bmj.com/cgi/content/full/337/nov11_2/a2390.
28. ^ a b c d Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ; CONSORT Group (2006).
"Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement". JAMA 295 (10): 1152–60. doi:
10.1001/jama.295.10.1152. PMID 16522836. http://jama.ama-assn.org/cgi/content/full/295/10/1152.
29. ^ a b c d e f g h i j k Schulz KF, Grimes DA (2002). "Generation of allocation sequences in randomised trials: chance, not choice". Lancet 359 (9305):
515–9. doi:10.1016/S0140-6736(02)07683-3. PMID 11853818. http://www.who.int/entity/rhl/LANCET_515-519.pdf.
30. ^ a b Lachin JM (1988). "Statistical properties of randomization in clinical trials". Control Clin Trials 9 (4): 289–311. doi:10.1016/0197-2456(88)90045-1.
PMID 3060315.

29. References
31. ^ Buyse ME (1989). "Analysis of clinical trial outcomes: some comments on subgroup analyses". Control Clin Trials 10 (4 Suppl): 187S–194S. doi:
10.1016/0197-2456(89)90057-3. PMID 2605967.
32. ^ a b c d e f Lachin JM, Matts JP, Wei LJ (1988). "Randomization in clinical trials: conclusions and recommendations". Control Clin Trials 9 (4): 365–74. doi:
10.1016/0197-2456(88)90049-9. PMID 3203526.
33. ^ Rosenberger WF, Lachin JM (1993). "The use of response-adaptive designs in clinical trials". Control Clin Trials 14 (6): 471–84. doi:10.1016/0197-2456(93)90028-C.
PMID 8119063.
34. ^ Forder PM, Gebski VJ, Keech AC (2005). "Allocation concealment and blinding: when ignorance is bliss". Med J Aust 182 (2): 87–9. PMID 15651970.
http://www.mja.com.au/public/issues/182_02_170105/for10877_fm.html.
35. ^ Pildal J, Chan AW, Hróbjartsson A, Forfang E, Altman DG, Gøtzsche PC (2005).
"Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study". BMJ 330 (7499): 1049. doi:
10.1136/bmj.38414.422650.8F. PMC 557221. PMID 15817527. http://www.bmj.com/cgi/content/full/330/7499/1049.
36. ^ a b c Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA (2008).
"Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study". BMJ 336 (7644): 601–
5. doi:10.1136/bmj.39465.451748.AD. PMC 2267990. PMID 18316340. http://www.bmj.com/cgi/content/full/336/7644/601.
37. ^ Devereaux PJ, Manns BJ, Ghali WA, Quan H, Lacchetti C, Montori VM, Bhandari M, Guyatt GH (2001).
"Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials". J Am Med Assoc 285 (15): 2000–3. doi:
10.1001/jama.285.15.2000. PMID 11308438. http://jama.ama-assn.org/cgi/content/full/285/15/2000.
38. ^ Haahr MT, Hróbjartsson A (2006). "Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors". Clin Trials 3 (4): 360–5. doi:
10.1177/1740774506069153. PMID 17060210. http://ctj.sagepub.com/cgi/content/abstract/3/4/360.
39. ^ Marson AG, Al-Kharusi AM, Alwaidh M, Appleton R, Baker GA, Chadwick DW, et al (2007). "The SANAD study of effectiveness of valproate, lamotrigine, or topiramate
for generalised and unclassifiable epilepsy: an unblinded randomised controlled trial". Lancet 369 (9566): 1016–26. doi:10.1016/S0140-6736(07)60461-9. PMC 2039891.
PMID 17382828.
40. ^ Chan R, Hemeryck L, O'Regan M, Clancy L, Feely J (1995).
"Oral versus intravenous antibiotics for community acquired lower respiratory tract infection in a general hospital: open, randomised controlled trial". BMJ 310 (6991):
1360–2. PMC 2549744. PMID 7787537. http://www.bmj.com/cgi/content/full/310/6991/1360.
41. ^ Fukase K, Kato M, Kikuchi S, Inoue K, Uemura N, Okamoto S, Terao S, Amagai K, Hayashi S, Asaka M; Japan Gast Study Group (2008). "Effect of eradication of
Helicobacter pylori on incidence of metachronous gastric carcinoma after endoscopic resection of early gastric cancer: an open-label, randomised controlled trial". Lancet
372 (9636): 392–7. doi:10.1016/S0140-6736(08)61159-9. PMID 18675689.
42. ^ Noseworthy JH, Ebers GC, Vandervoort MK, Farquhar RE, Yetisir E, Roberts R (1994).
"The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial". Neurology 44 (1): 16–20. PMID 8290055.
http://www.neurology.org/cgi/content/abstract/44/1/16.
43. ^ Manns MP, McHutchison JG, Gordon SC, Rustgi VK, Shiffman M, Reindollar R, Goodman ZD, Koury K, Ling M, Albrecht JK (2001). "Peginterferon alfa-2b plus ribavirin
compared with interferon alfa-2b plus ribavirin for initial treatment of chronic hepatitis C: a randomised trial". Lancet 358 (9286): 958–65. doi:
10.1016/S0140-6736(01)06102-5. PMID 11583749.
44. ^ Schwartz GG, Olsson AG, Ezekowitz MD, Ganz P, Oliver MF, Waters D, Zeiher A, Chaitman BR, Leslie S, Stern T; Myocardial Ischemia Reduction with Aggressive
Cholesterol Lowering (MIRACL) Study Investigators (2001).
"Effects of atorvastatin on early recurrent ischemic events in acute coronary syndromes: the MIRACL study: a randomized controlled trial". J Am Med Assoc 285 (13):
1711–8. doi:10.1001/jama.285.13.1711. PMID 11277825. http://jama.ama-assn.org/cgi/content/full/285/13/1711.
45. ^ a b Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, Jackson RD, Beresford SA, Howard BV, Johnson KC, Kotchen JM, Ockene J;
Writing Group for the Women's Health Initiative Investigators (2002).
"Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial".
J Am Med Assoc 288 (3): 321–33. doi:10.1001/jama.288.3.321. PMID 12117397. http://jama.ama-assn.org/cgi/content/full/288/3/321.

30. References
46. ^ Hollis S, Campbell F (1999). "What is meant by intention to treat analysis? Survey of published randomised controlled trials". Br Med J 319 (7211):
670–4. PMC 28218. PMID 10480822. http://www.bmj.com/cgi/content/full/319/7211/670.
47. ^ CONSORT Group. "Welcome to the CONSORT statement Website". http://www.consort-statement.org. Retrieved 2010-03-29.
48. ^ National Health and Medical Research Council (1998-11-16).
A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra: Commonwealth of Australia. p. 56. ISBN
1864960485. http://www.nhmrc.gov.au/_files_nhmrc/file/publications/synopses/cp30.pdf. Retrieved 2010-03-28.
49. ^ a b Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, Atkins D; Methods Work Group, Third US Preventive Services Task Force
(2001). "Current methods of the US Preventive Services Task Force: a review of the process". Am J Prev Med 20 (3 Suppl): 21–35. doi:
10.1016/S0749-3797(01)00261-6. PMID 11306229. http://www.ahrq.gov/clinic/ajpmsuppl/review.pdf.
50. ^ Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group (2008).
"What is "quality of evidence" and why is it important to clinicians?". BMJ 336 (7651): 995–8. doi:10.1136/bmj.39490.551019.BE. PMC 2364804. PMID
18456631. http://www.bmj.com/cgi/content/full/336/7651/995.
51. ^ Oxford Centre for Evidence-based Medicine (2009 March). "Levels of evidence". http://www.cebm.net/index.aspx?o=1025. Retrieved 2010-03-28.
52. ^ a b c Anderson JL, Pratt CM, Waldo AL, Karagounis LA (1997).
"Impact of the Food and Drug Administration approval of flecainide and encainide on coronary artery disease mortality: putting "Deadly Medicine" to the test"
. Am J Cardiol 79 (1): 43–7. doi:10.1016/S0002-9149(96)00673-X. PMID 9024734.
http://www.ajconline.org/article/S0002-9149%2896%2900673-X/abstract.
53. ^ a b Rubin R (2006-10-16). "In medicine, evidence can be confusing - deluged with studies, doctors try to sort out what works, what doesn't". USA
Today. http://www.usatoday.com/news/health/2006-10-15-medical-evidence-cover_x.htm. Retrieved 2010-03-22.
54. ^
"Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. The Cardiac Arrhythm
. N Engl J Med 321 (6): 406–12. 1989. doi:10.1056/NEJM198908103210629. PMID 2473403. http://content.nejm.org/cgi/content/abstract/321/6/406.
55. ^ Anderson GL, Limacher M, Assaf AR, Bassford T, Beresford SA, Black H, et al (2004).
"Effects of conjugated equine estrogen in postmenopausal women with hysterectomy: the Women's Health Initiative randomized controlled trial". JAMA
291 (14): 1701–12. doi:10.1001/jama.291.14.1701. PMID 15082697. http://jama.ama-assn.org/cgi/content/full/291/14/1701.
56. ^ Grodstein F, Clarkson TB, Manson JE (2003). "Understanding the divergent data on postmenopausal hormone therapy". N Engl J Med 348 (7): 645–
50. doi:10.1056/NEJMsb022365. PMID 12584376. http://content.nejm.org/cgi/content/extract/348/7/645.
57. ^ Vandenbroucke JP (2009). "The HRT controversy: observational studies and RCTs fall in line". Lancet 373 (9671): 1233–5. doi:
10.1016/S0140-6736(09)60708-X. PMID 19362661. http://linkinghub.elsevier.com/retrieve/pii/S0140-6736(09)60708-X.
58. ^ Hsu A, Card A, Lin SX, Mota S, Carrasquillo O, Moran A (2009).
"Changes in postmenopausal hormone replacement therapy use among women with high cardiovascular risk". Am J Public Health 99 (12): 2184–7. doi
:10.2105/AJPH.2009.159889. PMID 19833984. http://ajph.aphapublications.org/cgi/content/full/99/12/2184.
59. ^ a b c d e Black N (1996). "Why we need observational studies to evaluate the effectiveness of health care". BMJ 312 (7040): 1215–8. PMC 2350940.
PMID 8634569. http://www.bmj.com/cgi/content/full/bmj%3B312/7040/1215.
60. ^ a b c Sanson-Fisher RW, Bonevski B, Green LW, D'Este C (2007).
"Limitations of the randomized controlled trial in evaluating population-based health interventions". Am J Prev Med 33 (2): 155–61. doi:
10.1016/j.amepre.2007.04.007. PMID 17673104. http://www.ajpm-online.net/article/S0749-3797%2807%2900225-5/abstract.

31. References
61. ^ a b Rothwell PM (2005). "External validity of randomised controlled trials: "to whom do the results of this trial apply?"". Lancet 365 (9453): 82–93. doi:
10.1016/S0140-6736(04)17670-8. PMID 15639683. http://apps.who.int/rhl/Lancet_365-9453.pdf.
62. ^ Van Spall HG, Toren A, Kiss A, Fowler RA (2007).
"Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review". JAMA 297 (11):
1233–40. doi:10.1001/jama.297.11.1233. PMID 17374817. http://jama.ama-assn.org/cgi/content/full/297/11/1233.
63. ^ a b Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS (2006).
"Effect of a US National Institutes of Health programme of clinical trials on public health and costs". Lancet 367 (9519): 1319–27. doi:
10.1016/S0140-6736(06)68578-4. PMID 16631910. http://www.chrp.org/pdf/HSR20070511.pdf.
64. ^ a b Benson K, Hartz AJ (2000). "A comparison of observational studies and randomized, controlled trials". N Engl J Med 342 (25): 1878–86. doi:
10.1056/NEJM200006223422506. PMID 10861324. http://content.nejm.org/cgi/content/full/342/25/1878.
65. ^ a b Concato J, Shah N, Horwitz RI (2000). "Randomized, controlled trials, observational studies, and the hierarchy of research designs".
N Engl J Med 342 (25): 1887–92. doi:10.1056/NEJM200006223422507. PMC 1557642. PMID 10861325.
http://nejm.highwire.org/cgi/content/full/342/25/1887.
66. ^ Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, Contopoulos-Ioannidis DG, Lau J (2001).
"Comparison of evidence of treatment effects in randomized and nonrandomized studies". J Am Med Assoc 286 (7): 821–30. doi:
10.1001/jama.286.7.821. PMID 11497536. http://jama.ama-assn.org/cgi/content/full/286/7/821.
67. ^ Vandenbroucke JP (2008). "Observational research, randomised trials, and two views of medical science". PLoS Med 5 (3): e67. doi:
10.1371/journal.pmed.0050067. PMC 2265762. PMID 18336067.
68. ^ a b Glasziou P, Chalmers I, Rawlins M, McCulloch P (2007). "When are randomised trials unnecessary? Picking signal from noise". Br Med J 334
(7589): 349–51. doi:10.1136/bmj.39070.527986.68. PMC 1800999. PMID 17303884.
69. ^ Einhorn LH (2002). "Curing metastatic testicular cancer". Proc Natl Acad Sci U S A 99 (7): 4592–5. doi:10.1073/pnas.072067999. PMC 123692.
PMID 11904381.
70. ^ Bekelman JE, Li Y, Gross CP (2003). "Scope and impact of financial conflicts of interest in biomedical research: a systematic review".
J Am Med Assoc 289 (4): 454–65. doi:10.1001/jama.289.4.454. PMID 12533125. http://jama.ama-assn.org/cgi/content/full/289/4/454.
71. ^ a b Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ (2004).
"Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials". Can Med Assoc J
170 (4): 477–80. PMC 332713. PMID 14970094. http://ecmaj.com/cgi/content/full/170/4/477.
72. ^ Appelbaum PS, Roth LH, Lidz C (1982). "The therapeutic misconception: informed consent in psychiatric research". Int J Law Psychiatry 5 (3–4):
319–29. doi:10.1016/0160-2527(82)90026-7. PMID 6135666.
73. ^ a b Henderson GE, Churchill LR, Davis AM, Easter MM, Grady C, Joffe S, Kass N, King NM, Lidz CW, Miller FG, Nelson DK, Peppercorn J,
Rothschild BB, Sankar P, Wilfond BS, Zimmer CR (2007). "Clinical trials and medical care: defining the therapeutic misconception". PLoS Med 4 (11):
e324. doi:10.1371/journal.pmed.0040324. PMC 2082641. PMID 18044980.
74. ^ Wittes J (2002). "Sample size calculations for randomized controlled trials". Epidemiol Rev 24 (1): 39–53. doi:10.1093/epirev/24.1.39. PMID
12119854.
75. ^ Freiman JA, Chalmers TC, Smith H Jr, Kuebler RR (1978).
"The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials".
N Engl J Med 299 (13): 690–4. doi:10.1056/NEJM197809282991304. PMID 355881. http://content.nejm.org/cgi/content/abstract/299/13/690.

32. References
76. ^ Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P (2009-05-12). "Reporting of sample size calculation in randomised controlled trials: review". Br Med J 338:
b1732. doi:10.1136/bmj.b1732. PMC 2680945. PMID 19435763.
77. ^ Jain SL (2010). "The mortality effect: counting the dead in the cancer trial". Public Culture 21 (1): 89–117. doi:10.1215/08992363-2009-017.
78. ^ a b Farrington DP, Welsh BC (2005). "Randomized experiments in criminology: What have we learned in the last two decades?". Journal of Experimental Criminology 1
(1): 9–38. doi:10.1007/s11292-004-6460-0.
79. ^ Hollin CR (2008). "Evaluating offending behaviour programmes: does only randomization glister?". Criminology and Criminal Justice 8 (1): 89–106. doi:
10.1177/1748895807085871.
80. ^ Walker HM, Seeley JR, Small J, Severson HH, Graham BA, Feil EG, Serna L, Golly AM, Forness SR (2009). "A randomized controlled trial of the First Step to Success
early intervention. Demonstration of program efficacy outcomes in a diverse, urban school district". Journal of Emotional and Behavioral Disorders 17 (4): 197–212. doi:
10.1177/1063426609341645.
81. ^ Bradshaw CP, Zmuda JH, Kellam SG, Ialongo NS (2009). "Longitudinal impact of two universal preventive interventions in first grade on educational outcomes in high
school". Journal of Educational Psychology 101 (4): 926–937. doi:10.1037/a0016586.
82. ^ Duflo E, Glennerster R, Kremer M (2006-12-12). "Using randomization in development economics research: a toolkit" (PDF). MIT Department of Economics. Archived
from the original on 2007-01-19.
http://web.archive.org/web/20070119193234/http://www.povertyactionlab.com/papers/Using+Randomization+in+Development+Economics.pdf. Working Paper No. 06-36
83. ^ Banerjee AV, Cole S, Duflo E, Linden L (2007). "Remedying education: evidence from two randomized experiments in India". Quarterly Journal of Economics 122 (3):
1235–1264. doi:10.1162/qjec.122.3.1235.
84. ^ Karlan D, Zinman J (2010). "Expanding credit access: using randomized supply decisions to estimate the impacts". Review of Financial Studies 23 (1): 433–464.
doi:10.1093/rfs/hhp092.
85. ^ a b Olken BA (2007). "Monitoring corruption: evidence from a field experiment in Indonesia". Journal of Political Economy 115 (2): 200–249. doi:10.1086/517935.