CAN EFFECTIVENESS BE MEASURED OUTSIDE A CLINICAL TRIAL?

Transcription

1 CAN EFFECTIVENESS BE MEASURED OUTSIDE A CLINICAL TRIAL? Mette Nørgaard, Professor, MD, PhD Department of Clinical Epidemiology Aarhus Universitety Hospital Aarhus, Denmark

2

3

4

5 Danish Medical Birth Registry (since 1973)

6

7 EFFICACY, EFFECTIVENESS OF EFFICIENCY? Efficacy Does it works in clinical trials? Effectiveness Does it work in clinical practice? Efficiency Does it contribute to more efficient use of ressources?

8 DEFINITION OF COMPARATIVE EFFECTIVENESS The conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat, and monitor health conditions in real world settings. Federal Coordinating Council for Comparative Effectiveness Research

9

10 COMPARATIVE EFFECTIVENESS RESEARCH Research that identifies what clinical and public health interventions work best for improving health Not only direct clinical care such as diagnosis and treatment protocols, but also innovations in health care delivery, organization and financing, as well as public health interventions in the community

11 COMPARATIVE EFFECTIVENESS RESEARCH Research that identifies what clinical and public health interventions work best for improving health Not only direct clinical care such as diagnosis and treatment protocols, but also innovations in health care delivery, organization and financing, as well as public health interventions in the community

12 COMPARATIVE EFFECTIVENESS RESEARCH In a CER study, interventions should, at a minimum, be compared on the basis of some health-related outcome measure Study methods may include: randomized trials with at least two active intervention arms database studies observational studies model-based studies and decision analysis

13 EFFECTIVENESS AND CAUSALITY A crucial question is : What is the point of estimating a causal effect that is not well defined? The resulting relative risk estimate will not be helpful to either scientists, who will be unable to relate it to a mechanism, or policy makers, who will be unable to translate it into effective interventions

14 COUNTERFACTUAL CAUSALITY What would have happened to the same fixed individual at the same fixed time under one ( exposed ) versus another ( unexposed ) condition

15 COUNTERFACTUAL THINKING In counterfactual causal thinking we imagine the consequences of changing the value of a single factor in a comprehensive (complex) causal system

16 CAUSATION VERSUS ASSOCIATION A missing-data problem M A Hernán J Epidemiol Community Health 2004;58:

17 A CAUSAL CONTRAST OR TIME MACHINE NEEDED A causal contrast (or effect measure or causal parameter) compares disease experience under 2 exposure distributions but in 1 target population during 1 etiologic time period Small problem: time machine needed We use substitutes to enable comparison of a causal contrast (effect measure or causal parameter)

18 THE POTENTIAL OUTCOMES MODEL We can Substitute the time for the same population Substitute the population take a population other than the target population to represent the potential outcomes of target population Must assume exchangeability

19 RANDOMIZATION

20 RANDOMIZED CONTROLLED TRIALS Essential in efficacy and safety of treatments However, they do not provide information on pragmatic benefit in terms of major disease outcomes

21 AND The average randomized controlled trial is too small and does not have sufficient follow-up to detect adverse effects that are fewer than -one per 200 per year or that take longer than -one year to develop

22 RCT - PROBLEMS The 5 S they are often too: Selected (patients) Small (rare outcomes) Short (no long term effects/side effects) Simple (interactions) Specific (outcomes)

23 THE RANDOMIZED CONTROLLED TRIAL APPROACH 24

24 THE REAL-WORLD EVIDENCE APPROACH 25

25 OBSERVATIONAL STUDIES Nonexperimental observational studies may fill a gap by assessing longterm effect of treatments on infrequent outcomes Routinely collected administrative and health data + cautions on the methodological complexities of conducting observational studies

26 PRAGMATIC TRIALS Pragmatic trials are designed to evaluate the effectiveness of interventions in real-life routine practice conditions whereas explanatory trials aim to test whether an intervention works under optimal situations

27 Example

28 RANDOMIZATION Study population Patients with ST-segment elevation myocardial infarction (STEMI) who were undergoing percutaneous coronary intervention (PCI) Randomly assign treatment status Intervention = PCI + TA Comparator = PCI alone Count the deaths in 30 days following treatment Compare 30-day mortality in treated vs. untreated = causal contrast

29 Enrollment, Randomization, and Follow-up. FRÖBERT O ET AL. N ENGL J MED 2013;369:

30

31

32 OBSERVATIONAL FASE 4 STUDIES Cohort study Source population Exposed Unexposed Exposed Cases Unexposed Case-control study Control Group Exposed Unexposed Exposed Cases Unexposed

33

34

35

36 CONFOUNDING Confounding is a problem of comparison. We compare the exposed population to a substitute population. Confounding occurs if the substitute population does not show the outcome in the exposed population without the exposure

37 HANDLING OF CONFOUNDING Type of confounders Examples Strategy Measured Age, sex, Restriction Matching Stratification Standardization Regression analysis Unmeasured, but measurable Smoking Body Mass Index Disease severity Propensity Scores External adjustment Proxy measures Imputation Unmeasurable Frailty Self-controlled design Instrumental variable Mendelian randomization Active comparator Regression discontinuity design Sensitivity analyses

38

39 Does it really have to be this complicated? Can t I just trust my intuitions?

40 POTENTIAL FOR BIAS

41 IMMORTAL TIME BIAS

42 MULTIDISCIPLINARY TREATMENT A total of 6978 patients who were 66 years and older and had CKD A Cox model was used to determine the association between MDC and risk for death and hospitalization After adjustment for age, gender, baseline GFR, diabetes, and comorbidity score, there was a 50% reduction in the risk for death for the MDC compared with the non-mdc group hazard ratio = 0.50 (95% CI, 0.35 to 0.71). The benefits of MDC and an assessment of their economic impact should be tested in a randomized, controlled trial.

43 Salimah Z. Shariff et al. JASN 2008;19:

44 Kaplan-Meier survival curve, presented by Hemmelgarn et al.1. Salimah Z. Shariff et al. JASN 2008;19:

45 Immortal time bias. Salimah Z. Shariff et al. JASN 2008;19:

46 THE BIAS IN EPIDEMIOLOGY Immortal time refers to a span of time in the follow-up period in which the outcome under study could not have occurred usually passing of time before a subject initiates a given exposure

47 ANNALS INTERNAL MEDICIN 2001 Life expectancy was 3.9 years longer for Academy award winners than for other, less recognized performers 79.8 versus 75.8 years

48 LEAPING FROM ASSOCIATION TO CAUSATION: THE IMPOSSIBILITY OF PROOF Causal relation inferred by refuting alternative explanations of an association Uncertainty always present Fletcher & Fletcher, Clinical Epidemiology The Essentials

49 COMPARISON OF POOLED ODDS RATIO FROM OBSERVATIONAL STUDIES AGAINST POOLED ODDS RATIO FROM RANDOMIZED CONTROLLED TRIALS ON THE SAME QUESTION Ioannidis, J. P A et al. BMJ 2001;322:

50 ADR BURDEN To investigate adverse drug reactions either case-control or large-scale, observational follow-up studies are needed Vandenbroucke JP: When are observational studies as credible as randomised trials? Lancet. 2004;363:

51 CONCLUSION Effectiveness can be measured outside clinical trials

52 CONCLUSION (Pharmaco)epidemiology is not for amateurs

53 ANY QUESTIONS?

54