Pharmacoepidemiology Lecture: Designing Observational CER to Emulate an RCT
•Als PPTX, PDF herunterladen•
1 gefällt mir•926 views
Slides presented at http://www.verityresearch.org/verity-course-2020/ Q&A precedes the actual slides presented.
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Pharmacoepidemiology Lecture: Designing Observational CER to Emulate an RCT
Ähnlich wie Pharmacoepidemiology Lecture: Designing Observational CER to Emulate an RCT (20)
Mehr von Kazuki Yoshida
Mehr von Kazuki Yoshida (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Pharmacoepidemiology Lecture: Designing Observational CER to Emulate an RCT
- 1. Pharmacoepidemiology Lecture: Designing Observational CER to Emulate an RCT Kazuki Yoshida, M.D., Sc.D. Associate Epidemiologist Brigham and Women’s Hospital VERITY Online Course May 13th, 2020 1 @kaz_yos
- 2. Questions during lecture and my answers • Target trial emulation vs pragmatic trials? • Assume NO depletion of susceptibles, is prevalent user design ok? • What if most everybody in a dataset are prevalent users? • Definition of prevalent users given a dataset. What if initiation was close to the first encounter in the dataset? • New user design for effectiveness vs safety (particular long-term)? Any implications? • Violation of positivity assumptions and design principles? • How to account for medication adherence? Literature on this? • Active-comparator, new-user and time zero synchronization • TNFi vs non-TNFi: Can we really account for their differences? • Handling of missing data in observational CER? • Addressing immortal time issue with the target trial emulation framework. 2
- 3. Target trial emulation vs pragmatic trials? • Target trial emulation is a framework for observational studies that explicitly consider what the study is trying to do using clinical trial language. • A “target trial” is a conceptual hypothetical trial that you think about but don’t conduct. • Attached paper is the best place to start. 3
- 4. Assume NO depletion of susceptibles, is prevalent user design ok? • The new-user design is always better if feasible, just like RCT is better than observational studies if feasible. • Under some additional assumptions including no depletion of susceptible prior to database entry, prevalent user study may give a similar estimate. • The four discussion papers in the next slides helpful. Start with Hernan to avoid confusion 4
- 5. 5
- 6. What if most everybody in a dataset are prevalent users? • Check if the question is relevant. See Brookhart in the previous slide for sound alternatives to new-user design in the strict sense. • Note that the insight from the prevalent user study may not inform a decision directly. See Hernan in the previous slide. 6
- 7. Definition of prevalent users given a dataset. What if initiation was close to the first encounter in the dataset? • Inclusion of such “near new users” may be explicitly justified individual basis. • Is there left truncation of events? Is the lag period short enough to avoid having an outcome of interest (consider time-to-event setting) during that time? 7
- 8. New user design for effectiveness vs safety (particular long-term)? Any implications? • When the question is long-term safety, the relevant treatment decision may be what to do given a cohort of people who have already been on medication for some time (e.g., bisphosphonate). • Comparing prevalent users vs non-users do not inform this decision because it is not a decision relevant study (See Harnan). • Considering what is the treatment decision to be informed may be an important thing to start with (See Brookhart). 8
- 9. Violation of positivity assumptions and design principles? • Positivity holds if everyone is the observational study has some positive chance of receiving “other treatments” that they did not receive. • Positivity violation is existence of people with absolute indication to the treatment they received or contraindication for other treatments. • They would not appear in RCTs as they would no be appropriate to randomize. • Simply put, for those who cannot have received other treatment, the observational data does not contain information on treatment effect. 9
- 10. Positivity violation continued • In a non-initiator vs initiator study, this may be alleviated by carefully using pre-treatment covariates to include indications and exclude contraindications. • This may retain people with reasonable positive chance of hypothetically having received other treatment. • A brunt approach that pharmacoepidemiologists prefer is the active comparator design. Just compare people initiating one active treatment to another active treatment such that indications are ensured (e.g., RA disease activity) and contraindications are minimized (e.g., recent tuberculosis) 10
- 11. Positivity violation continued • Lund’s historical perspective for the active comparator design is useful. • Westreich gives a more formal treatment of positivity. 11
- 12. Positivity violation continued • The positivity assumption along with exchangeability and consistency constitutes what is called the “identifiability conditions”. • Identifiability conditions are theoretical conditions under which data can inform causal effects. • These conditions are ensured in RCTs (why they are gold standards). • They have to be assumed in observational CER in the end but can be improved by careful designs. • See Chapter 3 of the book (available free now) by Hernan & Robins. 12 https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/
- 13. How to account for medication adherence? Literature on this? • Definition of the treatment strategies (#2) is fundamental in defining what is considered “adherent to the strategy of interest”. • Causal contrast of interest (#6) then dictates whether we are interested in treatment effect with or without consideration of post- baseline adherence to treatment strategies. • In the target trial emulation framework, effects of interests are intention-to-treat (no consideration of adherence) and “per-protocol” (correction for adherence; note the use of this term varies by people and time). • The analysis plan (#7) for the “per-protocol” effect is necessarily more complicated. 13
- 14. Adherence continued • Note none of this is unique to observational CER (although likely more severe and difficult). • Thus, some literature trying to bridge the pragmatic trial (actual trials) and causal inference thinking developed in observational CER exists. • Eleanor Murray (@EpiEllie; Do follow her on Twitter)’s draft document (parts on intention-to-treat vs “per-protocol” in TTE sense) may be a good place to start. • https://www.hsph.harvard.edu/causal/pragmatictrials/ • Please note this is an active area of discussion. 14
- 15. Active-comparator, new-user and time zero synchronization • Getting time zero synchronization right as oppose to messing it up (Hernan et al) is simpler with the active-comparator, new-user design. • Danaei et al emulate three hypothetical trials. The non-initiator vs initiator (trial 1) is more complicated than the active comparator, new-user one (trial 3). 15
- 16. TNFi vs non-TNFi: Can we really account for their differences? • Let’s say we have managed to get new users only, observational studies are still confounded. • However, the degree of confounding by indication (also called channeling) can vary. • Frisell et al examined this in observational data. • They found 1st line TNFi vs non-TNFi is more confounded than subsequent TNFi vs non-TNFi, meaning later observational CER is “safer” (confounding adjustment is more plausible). 16
- 17. Handling of missing data in observational CER? • This is a huge methodological topic. Too huge to cover here. • In Zhao et al, this was the number 1 reason investigators cheat in the eligibility criteria (use post-baseline availability of data as “study eligibility”; no such thing in RCTs, which are prospective). • This is also not unique to observational CER although more severe. 17
- 18. Missing data continued • So we should probably learn from RCT literature as well. • There, the paradigm has shifted to ”serious handling of missing data” from “informal approaches” since the 2010 NRC report (https://www.ncbi.nlm.nih.gov/books/NBK209904/) • One take away is distinguishing real missing data (loss to follow-up) and ambiguous follow-up data points (e.g., CDAI measurement after initial bDMARD discontinuation). • Many observational CER studies (See Zhao) exclude the latter type of people (or censor these), but whether this is necessary depends on the treatment strategy definition and causal contrast of interest. 18
- 19. Missing data continued • If we are clear about defining treatment strategies (e.g. allowing for discontinuation after CDAI remission), it should be clear whether such treatment changes are compatible with the strategy (adherent) or not. 19
- 20. Addressing immortal time issue with the target trial emulation framework. • These two papers on the “clone-censor-ipw” approach may be of interest. 20
- 21. Pharmacoepidemiology Lecture: Designing Observational CER to Emulate an RCT Kazuki Yoshida, M.D., Sc.D. Associate Epidemiologist Brigham and Women’s Hospital VERITY Online Course May 13th, 2020 21 @kaz_yos
- 22. Pharmaco-epidemiology The study of the use of and effects of drugs in large numbers of people 22Strom et al. Pharmacoepidemiology. 6th ed.
- 23. One framework and two concepts 23 Zhao et al. Ann Rheum Dis. 2020 [Online First]. Yoshida et al. Nat Rev Rheumatol. 2015;11:437.
- 24. Key messages 24 Q https://www.flaticon.com/free-icon/database_660478 R Target trial Emulation • Study design issues are often result of emulation failure. • Specifying a target trial may prevent these. • Time zero synchronization: Eligibility, Assignment, & F/U Time 0. • Active-comparator design and new-user design can help.
- 25. Clinical Question 25 What is the best biological DMARD?
- 26. Pause: Best approach to CER and why? 26 R Randomization Reliable ascertainment of outcome Ability to encourage adherence Time-zero synchronization
- 27. Comparative effectiveness RCT example 27 R ≥ 18 y/o, RA ≥ 6 months, Prior use of MTX, but cannot continue No prior use of bDMARDs Gabay et al. Lancet 2013;381:1541. Initiate Tocilizumab 8mg/kg IV q4wk + ADA PBO Initiate Adalimumab 40mg SQ q2wk + TCZ PBO DAS28-ESR at 24wks Dose reduction, skipped doses, and discontinuation allowed for safety
- 28. Secondary-data observational CER 28 https://www.flaticon.com/free-icon/database_660478 https://favpng.com/png_view/insight-creativity-clip-art-png/ZmRiMZHV Months Not decade(s) Readily available observational data on bDMARD use Insight
- 29. Secondary-data observational CER 29 https://www.flaticon.com/free-icon/database_660478 What is the best biological DMARD? Careless design Vague question Readily available observational data on bDMARD use
- 30. Secondary-data observational CER 30 https://www.flaticon.com/free-icon/database_660478 What is the best biological DMARD? Careless design Vague question https://xkcd.com/882/ Readily available observational data on bDMARD use
- 31. Data from secondary DB and design from RCT 31 R https://www.flaticon.com/free-icon/database_660478 ♥️
- 32. Target trial emulation framework 32 Hernan & Robins. Am J Epidemiol. 2016;183:758.
- 33. Target trial emulation concept 33 Q https://www.flaticon.com/free-icon/database_660478 R Target trial Emulation
- 34. Target Trial Protocol Eligibility criteria Treatment strategies Assignment procedure Follow-up period Outcome Causal contrast of interest Analysis plan Target trial emulation implementation 34 Best bDMARD? R
- 35. Target Trial Protocol Eligibility criteria Treatment strategies Assignment procedure Follow-up period Outcome Causal contrast of interest Analysis plan Target trial emulation implementation 35 RWhich bDMARD to initiate among RA pts failing MTX?
- 36. Target Trial Protocol Eligibility criteria Treatment strategies Assignment procedure Follow-up period Outcome Causal contrast of interest Analysis plan http://www.clipartpanda.com/clipart_images/clipboard-clip-art-vector-59904421 Emulation Protocol Eligibility criteria Treatment strategies Assignment procedure Follow-up period Outcome Causal contrast of interest Analysis plan Target trial emulation implementation 36 RWhich bDMARD to initiate among RA pts failing MTX?
- 37. 37 Dickerman et al. Nat Med 2019;25:1601. Side note: Rheum examples in making [unpublished] Houchen Lyu et al. Vibeke Norvang et al.
- 39. Design elements of a target trial 39 R
- 40. 1. Eligibility criteria 40 R 1. Eligibility criteria ≥ 18 y/o, RA ≥ 6 months, Prior failure of MTX No prior use of bDMARDs
- 41. 2. Treatment strategies 41 R 2. Treatment strategies Continue MTX and initiate tocilizumab Continue MTX and initiate adalimumab 1. Eligibility criteria ≥ 18 y/o, RA ≥ 6 months, Prior failure of MTX No prior use of bDMARDs Continue bDMARD unless severe adverse events develop. Discontinuation for remission allowed. Switching bDMARD not allowed.
- 42. 3. Treatment assignment procedure 42 R 3. Tx assignment 2. Treatment strategies Continue MTX and initiate tocilizumab Continue MTX and initiate adalimumab 1. Eligibility criteria ≥ 18 y/o, RA ≥ 6 months, Prior failure of MTX No prior use of bDMARDs Continue bDMARD unless severe adverse events develop. Discontinuation for remission allowed. Switching bDMARD not allowed.
- 43. 4. Follow-up and 5. Outcome 43 R 3. Tx assignment 4. Follow-up period e.g., 1 year 5. Outcome e.g., CDAI at 1 year 2. Treatment strategies Continue MTX and initiate tocilizumab Continue MTX and initiate adalimumab 1. Eligibility criteria ≥ 18 y/o, RA ≥ 6 months, Prior failure of MTX No prior use of bDMARDs Continue bDMARD unless severe adverse events develop. Discontinuation for remission allowed. Switching bDMARD not allowed.
- 44. 6. Causal contrasts and 7. Statistical analysis 44 R 3. Tx assignment 4. Follow-up period e.g., 1 year 5. Outcome e.g., CDAI at 1 year 2. Treatment strategies Continue MTX and initiate tocilizumab Continue MTX and initiate adalimumab 1. Eligibility criteria ≥ 18 y/o, RA ≥ 6 months, Prior failure of MTX No prior use of bDMARDs Continue bDMARD unless severe adverse events develop. Discontinuation for remission allowed. Switching bDMARD not allowed. 6. Causal contrasts Intention-to-treat Per-protocol 7. Statistical Analysis ITT: Ignore deviations PP: Censor deviations and adjust for selection
- 45. Time zero synchronization 45 Time Zero - Eligibility met - Tx assigned - F/U starts R 3. Tx assignment 4. Follow-up period e.g., 1 year 5. Outcome e.g., CDAI at 1 year 2. Treatment strategies Continue MTX and initiate tocilizumab Continue MTX and initiate adalimumab 1. Eligibility criteria ≥ 18 y/o, RA ≥ 6 months, Prior failure of MTX No prior use of bDMARDs Continue bDMARD unless severe adverse events develop. Discontinuation for remission allowed. Switching bDMARD not allowed. 6. Causal contrasts Intention-to-treat Per-protocol 7. Statistical Analysis ITT: Ignore deviations PP: Censor deviations and adjust for selection
- 46. How can things go wrong? 46 Hernan et al. J Clin Epi 2016;79:70. Zhao et al. Ann Rheum Dis. 2020 [Online First].
- 48. Prevalent user design issue 48 1 2 3 4 5 6 Subject # Time axis High risk period early on treatment Medium risk period Subsequent lower risk period Time period off treatment
- 49. Prevalent user design issue 49 Prevalent user design F/U Time Zero 1 2 3 4 5 6 Subject # 1 2 3 4 5 6 Time axis High risk period early on treatment Medium risk period Subsequent lower risk period Time period off treatment
- 50. Prevalent user design issue: Emulation failure 50 R F/U time zero F/U time zero
- 51. Prevalent user design issue: New-user design 51 New user design Prevalent user design F/U Time Zero F/U Time Zero 1 2 3 4 5 6 Subject # 1 2 3 4 5 6 2 3 6 Time axis High risk period early on treatment Medium risk period Subsequent lower risk period Time period off treatment Ray. Am J Epidemiol. 2003;158:915. ╳
- 52. Pause: New-user design: pros and cons? 52
- 53. Non-user comparator issue 53 R 1. Eligibility criteria: Common to all arms. 3. Tx assignment: Positive probability for everyone. Time Zero - Eligibility met - Tx assigned - F/U starts
- 54. Non-user comparator issue 54 Measured pretreatment characteristics: age, gender, etc. (These can be adjusted statistically.) Unmeasured pretreatment characteristics: frailty, disease activity (in claims data), etc. (These cannot be addressed statistically.) Initiators of TNFi Non-users
- 55. Non-user comparator vs active comparator 55 Measured pretreatment characteristics: age, gender, etc. (These can be adjusted statistically.) Unmeasured pretreatment characteristics: frailty, disease activity (in claims data), etc. (These cannot be addressed statistically.) Initiators of TNFi Initiators of non-TNFi Non-users
- 56. 56Kim et al. Arthritis Rheumatol. 2017;69:1154
- 57. Pause: Active comparator: pros and cons? 57
- 58. Active comparator, new user design 58Yoshida et al. Nat Rev Rheumatol. 2015;11:437. Lund et al. Curr Epidemiol Rep. 2015;2:221.
- 59. ACNU and its relation to target trial emulation •Active comparator •Similar indications/contraindications •Easier time synchronization •New user •Time zero synchronization •More justifiable confounding adjustment 59
- 60. Key messages 60 Q https://www.flaticon.com/free-icon/database_660478 R Target trial Emulation • Study design issues are often result of emulation failure. • Specifying a target trial may prevent these. • Time zero synchronization: Eligibility, Assignment, & F/U Time 0. • Active-comparator design and new-user design can help.
- 61. Back up slides follow 61
- 62. 62http://cerbot.org Comparative Effectiveness Research Based on Observational Data to Emulate a Target Trial CERBOT is a tool for improving comparative effectiveness research using observational data. CERBOT was designed for researchers and clinicians with basic statistical training to work as part of a team with patients and other stakeholders.
- 63. Time lag bias issue 63
- 64. Immortal time bias issue 64 Iudici et al. Ann Rheum Dis. 2019;78:562.
- 65. Design components of target trial emulation 65 R Protocol component Target trial protocol Emulation protocol
- 66. 1. Eligibility criteria 66 Protocol component Target trial protocol Emulation protocol 1. Eligibility criteria Age ≥ 18 at baseline Clinically diagnosed RA ≥ 6 months Still being treated with MTX No previous use of bDMARDs No contraindications for adalimumab or tocilizumab (e.g., tuberculosis, cancer within 1 year, etc) Same as for the target trial Physician diagnosis of RA MTX prescription within last 3 months ≥ 1 year of baseline data before baseline R
- 67. 2. Treatment strategies 6767 Protocol component Target trial protocol Emulation protocol 2. Treatment strategies (1) Continue MTX and initiate tocilizumab IV q4wk (2) Continue MTX and initiate adalimumab SQ Continue bDMARD unless severe adverse events (e.g., infections) Discontinuation for remission allowed Switch to another bDMARD not allowed Same as for the target trial (1) Delay in IV < 4wks assumed continuation (2) Rx gap < 30 days assumed continuation for adalimumab Longer gaps are considered discontinuation at 4wks / 30 days D/C with record of remission allowed IV or Rx of other bDMARDs considered switch R
- 68. 3. Treatment assignment 6868 Protocol component Target trial protocol Emulation protocol 3. Treatment assignment Individuals are randomly assigned to either treatment strategy and will be aware of the assignment Individuals are assigned to the treatment strategy based on first bDMARD after ≥ 1 year baseline without bDMARD Note the initiation of bDMARD other than tocilizumab or adalimumab constitutes baseline ineligibility R
- 69. 4. Follow-up and 5. Outcome 69 Protocol component Target trial protocol Emulation protocol 4. Follow-up Starts at baseline and ends at 12 months or loss to follow-up Same as for the target trial 5. Outcome Clinical Disease Activity Index at 12 months (± 1 month) Same as for the target trial R
- 70. 6. Causal contrasts and 7. Statistical analysis 70 Protocol component Target trial protocol Emulation protocol 6. Causal contrasts (1) Intention-to-treat effect (effect of treatment assignment) (2) Per-protocol effect (effect under full adherence to strategies) (1) Observational analog of intention-to- treat effect (effect of treatment initiation) (2) Observational analog of per-protocol effect (effect under full adherence to strategies) 7. Statistical analysis (1) ITT: Unadjusted (no baseline confounding); need adjustment for loss to follow-up (2) PP: + Artificially censor strategy deviations; adjust with inverse probability of censoring (1) ITT analog: Adjust for baseline confounding by indication; need adjustment for loss to follow-up (2) PP analog: + Artificially censor strategy deviations; adjust with inverse probability of censoring R
Hinweis der Redaktion
- Hello everyone, This is the pharmacoepidemiology lecture in the VERITY Research Seminar Series. I am Kazuki Yoshida, an Associated Epidemiologist at the Brigham and Women's Hospital.
- Hello everyone, This is the pharmacoepidemiology lecture in the VERITY Research Seminar Series. I am Kazuki Yoshida, an Associated Epidemiologist at the Brigham and Women's Hospital.
- What is pharmaco-epidemiology? It is defined as the the study of the use of and effects of drugs in large numbers of people, as you can see in the canonical textbook in pharmacoepidemiology. The scope of pharmacoepidemiology is broadening in recent years, we can understand it as the use of the techniques of chronic disease epidemiology to study the use of and the effects of treatment strategies. In this talk, we will specifically focus on the use of epidemiological principles to guide the design of observational comparative effectiveness research (CER).
- I will cover one study design framework and two study design principles. The CER design framework that I will introduce is called the “target trial emulation framework”. The two pharmacoepidemiology design principles that are useful in operationalizing target trial emulation are the "active-comparator design" and the "new-user design".
- In this presentation, I'll use a cloud to indicate the clinical question, a fork diagram to indicate a hypothetical RCT that would answer the question, and a stack diagram to indicate an observational dataset, which we can use. I'll state the key phrases up front. - emulation failure - specifying a target trial - time zero synchronization - active-comparator design and new-user design
- Let’s say there is a clinical comparative effectiveness question: What is the best biological DMARD? Putting aside its vagueness for now, what is the best approach to answer this comparative effectiveness question?
- What is the best approach to a comparative effectiveness question and why? A randomized controlled trial (RCT) is the best approach to address any comparative effectiveness question — if an RCT is indeed an option. The benefit may include obvious ones to subtle ones.
- For example, the ADACTA trial enrolled RA patients intolerant of methotrexate, and randomized them to initiation of tocilizumab or adalimumab combined with their respective placebos and examined DAS28-ESR at 24 weeks. It demonstrated the superiority of tocilizumab monotherapy in comparison to adalimumab monotherapy in this specific RA patient population. Ideally, we want this type of comparative effectiveness RCT for every patient population of interest and every choice of treatment strategy. However, there are relatively few of these. Additionally, by design, RCTs have limited sample sizes and duration to examine non-primary endpoints such as long-term safety. When RCTs are not feasible, timely, or ethical, observational data may help fill the gap in comparative effectiveness information (Quote from Zhao et al 2020).
- We increasingly have routinely collected data that can be repurposed for research use. For example, insurance claims and more recently electronic health records (EHRs) are typical examples. Additionally, some research purpose registries can be used for purposes beyond the original aims. The availability of secondary data is helpful in that the execution of research may be faster and less resource intensive than the corresponding RCTs. The insight from data may come in months, instead of a decade or two.
- The danger of readily available data is that we can go ahead with a vague question and a careless design, producing non-interpretable results.
- Even worse, there is almost no penalty for repeating the process until we get the “green jelly beans” result.
- We tend to think of observational CER and RCTs as opposing concepts, completely different world with entirely different research practices. However, in conducting observational CER, we should really think how ideas from these two world can be combined. That is, the study design principles in the RCT world can help us design and conduct better observational CERs. This is not an entirely new idea and dates back to 1950’s.
- More recently, researchers at the Harvard T.H. Chan School of Public Health took this notion of borrowing design ideas from RCT to conduct and observational CER to the extreme. They started calling it “target trial emulation”.
- The main idea of target trial emulation is simple. If we have a comparative effectiveness question, we should think of an ideal RCT that we would like to conduct. This is called the target trial, the trial we really want to see. Given the opportunity, we should conduct this. If not, we may want to consider how we can use available observational data to emulate such a target trial. That is, to conduct an observational study designed in a way that tries to answer the same question that the target trial would answer.
- The key to implementation of this idea is to actually write it down as protocol outlines. Hernan and Robins’ 2016 paper recommended a minimum of 7 items: eligibility criteria, treatment strategies of interest, assignment procedure, follow-up period, outcome, causal contrast of interest, and corresponding analysis plan. This is essentially an expanded version of the familiar PICO: Patients, Intervention, Control [intervention], and Outcome. This forces us to think of (1) implementable treatment strategies and (2) testable hypothesis. This is not necessarily a one way process. By constructing a protocol of target trial concretely, it may improve the question we are posing.
- A more refined version of the question may be: Among MTX insufficient responder RA patients without previous bDMARD use but not intolerant of MTX, which bDMARD should be initiated to reduce disease activity in 1 year.
- Only after we have a reasonably concrete idea of the target trial, clear enough to write down, we can think of how observational data may help. We can formulate the same items for the emulation protocol. The emulation protocol outlines how we emulate these items in the target trial protocol in an observational dataset. The proposed benefit of this two-step approach is that we automatically follow the good study design principles and identify and avoid unnecessary sources of bias that are common in observational comparative effectiveness research.
- The protocols can be summarized in a table format. Explicit target trial emulation is a relatively new framework and there are not that many examples. In the current literature, this 2019 paper by Barbra Dickerman and colleagues may be the clearest example. You may see rheumatology examples in coming months from Houchen Lyu and Vibeke Norvang. This table here uses the same seven items.
- The middle column outlines the target trial protocol, the protocol for the ideal RCT that they really wanted. The right column describes how these items are operationalized in emulating the target trial with the observational data available.
- What are the design elements of an RCT that may guide us in emulation? Here we examine several elements of the target trial and their implications to the emulation with observational data.
- Firstly, we need a set of eligibility criteria based on pre-treatment patient characteristics. We must ensure the eligible patients have indications for all study treatment strategies. In emulating this observational data, we also must make sure that we use pre-treatment patient characteristics for eligibility. To the extent possible, patient characteristics that may pose contraindication, for example, recent cancer, should be handled as exclusion criteria.
- We need to clearly define treatment strategies of interest. We say strategies because clinical care is not just about the choice of drug, but also about how to use it over time. In an RCT, the study protocol specifies what treatment modifications are allowed or not. In an observational study emulating this, defining treatment strategies means departing from the notion of comparing the exposed and the unexposed. We should instead think in terms of actions to be implemented. That is, we are talking about actions to be taken from now on rather than talking about exposures that have happened.
- In an RCT, treatment strategies are assigned randomly. Importantly, this ensures that everyone has a positive chance to be assigned to any of the study treatment strategies. In observational data from clinical practice, assignment to treatment strategies are carefully chosen based on pre-treatment information on prognosis, departing from random allocation. In emulating step, we need to somehow attempt to recover the random assignment. This mean we need to collect data an all prognostic factors that may have driven the treatment decision in the observational data and statistically control for them.
- In an RCT, it is always necessary to define the follow-up period. Outcomes as well as ascertainment must be specified. In emulating these, we should provide similar definitions and how they are operationalized in a given dataset.
- The causal contrasts of interest in an RCT includes the intention-to-treat effect, which is the effect of being assigned to a treatment strategy vs another. In the target trial emulation framework, the "per-protocol effect" means the effect of the treatment strategy under full adherence. The statistical analysis for the ITT effect ignores the treatment strategy deviations that occurred after baseline. The analysis for the per protocol effect censor individuals at the time of deviation and use statistical techniques to adjust for resulting selection bias. In an observational study trying to emulate these, the definitions of effects is in terms of initiation of treatment instead of intention. Statistical analyses are conceptually the same except for the additional need for adjusting for baseline confounding, which does not exist in RCTs.
- Implicit in this diagram is the natural synchronization of time zero in RCTs. The eligibility is met at the time of treatment strategy assignment, and the follow-up for the outcome starts immediately. All three elements meet in the middle of the diagram. In emulating this, we should ensure that eligibility criteria use strictly pre-treatment information without reference to future data, such as continued use bDMARDs in 1 year. Similarly, treatment assignment must occur once eligibility is met. Follow-up for outcomes must start immediately.
- How can things go wrong in observational CER when we do not have the target trial emulation framework? These papers examined this question. The first paper by Hernan and colleagues explain time-related biases can easily occur when we do not clarify the target trial. In the second paper, Steven Zhao looked at potential sources of bias in existing rheumatology observational CER papers that could have been prevented by explicit target trial emulation.
- The key here is that typical study design flaws encountered in observational studies are often result of failures to specify a clear target trial and to explicitly emulate it. In the following, we will review some study design issues in terms of their target trial emulation failures.
- In pharmacoepidemiology, prevalent users mean those who have been on the treatment strategy of interest prior to the start of the follow up and still on it. For example, in studying bDMARDs using EHR, inclusion of people who have been using the same TNFi in outside care constitutes a prevalent user design. In the diagram, each line is a patient, the thicker part is the part of follow up following a treatment strategy, e.g., TNFi use. The gray part is not captured in the EHR.
- In prevalent user design, the follow-up time zero is set to the time of treatment initiation for those who happened to have the treatment initiation, for example, bDMARD, in the database. For those without the initial part of treatment history, the follow-up time zero is set to the arbitrary time point after treatment initation.
- The prevalent user design fails to emulate a sound target trial. The corresponding RCT would randomize patients and initiate treatment strategies. At some point after randomization, select the subgroup of patients who are still alive and adherent to the treatment in each arm, and start the follow-up there setting time to zero. Intuitively, this can exclude early adverse events in a safety study, potentially affecting the result. Another way to understand this emulation failure is that we cannot randomize people to a new state “having been on TNFi for 1 years”.
- The solution here is the new-user design. In new user design, we start the follow up at the time of treatment initiation for everyone included. Those who do not have Therefore, it provides better time zero synchronization. A more detailed argument typically includes presence of early events right after treatment strategy initiation. Such early events may be missed in the prevalent users who have already survived such high risk periods, artificially lowering the risk estimate. [fix the note]
- In an RCT, patients must be eligible to all the treatment strategies of interest because we cannot include patients who should not be randomized. This is ensured in an RCT by common eligibility criteria that include patients with indications and exclude patients with contraindications. Careless use of non-user comparators in an observational CER can violate this principle. Those without indication or with contraindication may sneak into the non-user arm of the study. To emulate randomization, we need to have access to pre-treatment covariates that are relevant to treatment decisions. They can be statistically addressed if they are measured. Additionally, time zero synchronization tends to be harder for non-users.
- To emulate randomization, balancing both measured and unmeasured baseline patient characteristics is necessary. In this iceberg analogy, statistical techniques such as propensity score methods can account for measured characteristics above the surface, but not the hidden unmeasured characteristics below the surface.
- Choosing an active comparator carefully can benefit the balance of covariates beyond those are directly measured in the datataset. In this example, initiators of non-TNFi bDMARDs are expected more similar to initiators of TNFi than non-users, not only in the measured characteristics, but also in unmeasured characteristics that may have influenced the treatment patterns observed. This may also benefit the correspondence to the real practice. Given a set of RA patients who are candidate for TNFi, the alternative strategy of interest is likely not withholding treatment.
- Here is an example of patient baseline characteristics in a well conducted active-comparator study. There are three datasets. Within each the tocilizumab group and the comparator TNFi are well balanced.
- Active-comparator, new-user design CER can be thought as an attempt to emulate a head-to-head target trial. Our review in Nature Reviews Rheumatology covers the basics using rheumatology examples. The review by Lund and colleagues gives a more detailed account of these principles, emphasizing their role in emulating a head-to-head target trial.
- In summary, the active comparator ensures similar indications and contraindications. This helps ensuring a sound eligibility criteria (#1 in TTE), treatment strategies that are important in comparing (#2 in TTE). By having two strategies that are both initiated, it also helps time zero synchronization, which can be trickier for non-user comparison without a clear reference time point. The new user aspect is the key to time zero synchronization. Additionally, by making sure we have pre-treatment covariates available for confounding adjustment, emulation of random treatment assignment can be improved.
- [Animate after stating all] Question → Target Trial → Emulation Typical study design issues are often result from emulation failure. Specifying and explicitly emulating a target trial may prevent these. The most fundamental aspect of target trial emulation is time zero synchronization. Synchronization of the time eligibility is met, treatment strategies are assigned, and follow-up for outcome starts. Active-comparator, new-user design can help enforce an emulation of a head-to-head target trial.
- What are the design elements of an RCT that may guide us in emulation? Here we examine several elements of the target trial and their implications to the emulation with observational data, which we pretend as a somewhat idealized EHR data source.
- Firstly, we need a set of eligibility criteria based on pre-treatment patient characteristics. We must ensure the eligible patients have indications for all study treatment strategies. In emulating this observational data, we also must make sure that we use pre-treatment patient characteristics for eligibility. To the extent possible, patient characteristics that may pose contraindication, for example, recent cancer, should be handled as exclusion criteria.
- We need to clearly define treatment strategies of interest. We say strategies because clinical care is not just about the choice of drug, but also about how to use it over time. In an RCT, the study protocol specifies what treatment modifications are allowed or not. In an observational study emulating this, defining treatment strategies means departing from the notion of comparing the exposed and the unexposed. We should instead think in terms of actions to be implemented. That is, we are talking about actions to be taken from now on rather than talking about exposures that have happened.
- In an RCT, treatment strategies are assigned randomly. Importantly, this ensures that everyone has a positive chance to be assigned to any of the study treatment strategies. In observational data from clinical practice, assignment to treatment strategies are carefully chosen based on pre-treatment information on prognosis, departing from random allocation. In emulating this step, we need to somehow attempt to recover the random assignment. This mean we need to collect data an all prognostic factors that may have driven the treatment decision in the observational data and statistically control for them.
- In an RCT, it is always necessary to define the follow-up period. Outcomes as well as ascertainment must be specified. In emulating these, we should provide similar definitions and how they are operationalized in a given dataset.