Garvan Ins)tute Biosta)s)cal Workshop 7/5/2015. Tuan V. Nguyen. Garvan Ins)tute of Medical Research Sydney, Australia

Transcription

1 Garvan Ins)tute Biosta)s)cal Workshop 7/5/2015 Tuan V. Nguyen Tuan V. Nguyen Garvan Ins)tute of Medical Research Sydney, Australia

2 What I am going to cover... Randomized controlled trial (RCT) Concept of "risk" Interpreta)on of RCT data

3 Evidence based medicine vocabulary Efficacy RR NNH EFFECT SIZE OR 95% CI RRR Cost-benefit Cost-effective Time-to-event analysis Intention-to-treat Sensitivity ARR P-value Risk versus benefit Meta-analysis DB-PC-RCT HR Likelihood ratio Population attributable risk

4 Randomized controlled trial

5 Randomized controlled trial An experiment in which subjects are randomly allocated into groups, usually called study and control groups, to receive or not to receive an experimental preven)ve or therapeu)c procedure, maneuver, or interven)on. The results are assessed by rigorous comparison of rates of disease, death, recovery, or other appropriate outcome in the study and control groups, respec)vely.

6 Randomized controlled trial An important advance in the history of medical science Gold standard for evalua)ng efficacy (and safety?) of an interven)on Allow cause- effect inference (But it is probably the worst method for evalua)ng an individual pa)ent)

7 Randomized controlled trial First RCT: use of streptomycin in the treatment of TB Sir Bradford Hill and Sir Richard Doll

8 Photograph courtesy of Ernie Branson. Randal J JNCI J Natl Cancer Inst 1999;91:10-12 Austin Bradford Hill

9 The main idea of RCT Rx Randomized into groups Follow-up Comparison of outcome Placebo

10 The "risk" concept

11 "Risk factor" A concept originated from the Framingham Heart Study (Dr. William Kannel in 1961, Ann Int Med) Risk factor a measurable characteris)c causally associated with increased disease frequency The concept changed the way medicine is prac)sed

12 Risk Risk Probability of adverse and undesirable event during a period Characteris)cs Probability Time Undesirable event

13 Disease and risk of disease Disease a binary trait (yes/no) applicable to an individual Risk of disease a con)nuous trait (0 1) applicable to a group of individuals

14 Hazard Hazard can be confusing! Short: A hazard is the rate at which events occur Long: A hazard (H) is the ra)o of the probability of an event occurring in a short )me interval (P) over the length of :me (L) In other words, P = H x L

15 An example: Dubbo Osteoporosis Epidemiology Study guyen et al., JBMR women aged 60+ Osteoporosis 345 (27%) Fracture: 137 Risk: 137/345 = 0.40 Non-osteoporosis 942 (73%) Fracture: 191 Risk: 191/942 =

16 Odds Odds - a risk- related concept Odds = ra)o of the probability of occurrence of an event to that of non- occurrence If P is the risk of fracture, the odds of fracture is defined as odds = P 1 P

17 Odds is not risk!

18 Metrics of effect in RCT

19 Metrics of effect in RCT RelaFve metrics Rela)ve risk (risk ra)o) Odds ra)o Hazards ra)o Absolute metrics Absolute risk difference Number needed to treat (NNT)

20

21 Results (year 5 data) Outcome E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Stroke 16 8 Hip fracture 5 8 Death How do we assess the effect? WHI, JAMA 2002; 288:

22 WHI data risk of cancer Outcome E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Stroke 16 8 Hip fracture 5 8 Death Risk1 = = Risk 0 = =

23 Risk ratio (RR) Commonly referred to as Rela:ve Risk Defini)on of RR Risk1 RR = = Risk 0 Risk Risk ( Rx) ( control )

24 Meaning of RR Risk1 RR = = Risk 0 Risk Risk ( Rx) ( control ) RR = 1, there is no effect RR < 1, beneficial effect RR > 1, harmful

25 WHI data breast cancer RR Outcomes E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Stroke 16 8 Hip fracture 5 8 Death Risk1 = = Risk 0 = = RR = = 2.59

26 Doctors say: 1% breast cancer risk PaFent hears: 99% chance of no breast cancer

27 WHI data an alternative expression Outcomes E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Risk of inv breast cancer Risk of NON- cancer RR = = It seems that there was no effect!

28 Introduction to odds ratio Risk ra)o is a asymmetric measure Odds ra)o (OR) = ra)o of two odds OR is symmetric (nice property!)

29 Odds ratio (OR) Odds1 = odds of event in the Rx group Odds0 = odds of event in the control group OR = Odds 1 Odds 0 OR = 1, there is no effect OR < 1, beneficial effect OR > 1, harmful

30 An example of OR Outcomes E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Risk of inv breast cancer (P) Risk of NON- cancer (1- P) Odds: P / (1- P) OR = = 2.65

31 An example of OR demonstration of symmetry Outcomes E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Risk of inv breast cancer (P) Risk of NON- cancer (1- P) Odds: (1- P) / P OR = = 2.65

32 A comparison between RR and OR RR = 2.59: Rela)ve to placebo, E+P increased the risk of invasive breast cancer by 2.6 )mes OR = 2.65: Rela)ve to placebo, E+P increased the odds of invasive breast cancer by 2.6 )mes Risk is not odds!

33 But be careful with odds ratio! OR can exaggerate an effect, par)cularly in studies with common events Consider the following "discrimina)on study"

34 Recommendation for catheterization? Figure 1. Patients as Portrayed by Actors in the Video Component of the Survey. Panel A shows a 55-year-old black woman, Panel B a 55-year-old black man, Panel C a 70- year-old black woman, Panel D a 70-yearold black man, Panel E a 55-year-old white woman, Panel F a 55-year-old white man, Panel G a 70-year-old white woman, and Panel H a 70-year-old white man.

35 Race, sex, and catheterization Catheterization No Catheterization Total White Black Schulman et al. New England Journal of Medicine (25/2/1999 ) Logistic regression analysis indicated that women (odds ratio, 0.60; 95 percent confidence interval, 0.4 to 0.9; P=0.02) and blacks (odds ratio, 0.60; 95 percent confidence interval, 0.4 to 0.9; P=0.02) were less likely to be referred for cardiac catheterization than men and whites, respectively. Analysis of race sex interactions showed that black women were significantly less likely to be referred for catheterization than white men (odds ratio, 0.4; 95 percent confidence interval, 0.2 to 0.7; P=0.004).

36

37 Where is the truth? Catheterization No catheterization Black White OR = = 0.59 p black = = p white = 652 = PR = = 0.94 WHY?

38 Relationship between OR and RR Odds ra)o is an es#mate of rela)ve risk Event No event Group 1 a b Group 2 c d RR = a a+ b c c+ d When a and b are small (relative to c and d) a RR ; b ad ; ; c bc OR d

39 Odds ratio overestimates relative risk Study Risk of disease Odds of disease Comparison between RR and OR Group 1 (p 1 ) Group 2 (p 2 ) Nhóm 1 (odd 1 ) Nhóm 2 (odd 2 ) RR OR

40 For a constant RR, OR increases with background risk Odds ratio Disease incidence/prevalence

41 Absolute metrics

42 Problem with RR: background risk Study 1 Study 2 Placebo Rx Placebo Rx N Disease Risk (%) RR

43 Absolute risk reduction P 1 : risk of disease in Rx P 0 : risk of disease in controls ARR = P 1 P 2

44 WHI data Absolute difference Outcomes E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Stroke 16 8 Hip fracture 5 8 Death Risk1 = = Risk 0 = = ARR = =

45 Absolute Risk Reduction ARR = Difficult to understand! Hard to convey to pa)ents (and the public)

46 CMAJ Analysis The number needed to treat turns 20 and continues to be used and misused Finlay A. McAlister MD MSc CMAJ SEPTEMBER 9, (6) 2008 Canadian Medical Association or its licensors In the 20 years since the initial description Key of the points number The number needed to treat is a useful measure for counselling patients about their potential to benefit from a particular intervention. It is sometimes used as a basis for comparing 2 or more therapies; however, it is important to appreciate that this number is not therapy-specific, but rather it is specific to the results of a single comparison. If it is to be used to compare treatments, the therapies must have been tested in similar populations with the same condition at the same stage, using the same comparator, time period and outcomes. The factors that influence the number needed to treat beyond the efficacy of the treatment must be taken into account to avoid drawing erroneous conclusions when comparing numbers needed to treat for 2 or more interventions.

47 NNT number needed to treat An indea of Andreas Laupacis (Laupacis A, Sackeq DL, Roberts RS (1988). "An assessment of clinically useful measures of the consequences of treatment". N. Engl. J. Med. 318 (26): ) Number of pa)ents need to be treated in order for one to benefit Defini)on of NNT: NNT = 1 ARR = Risk 1 1 Risk 0

48 WHI data NNT (hip fracture) Outcome E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Stroke 16 8 Hip fracture 5 8 Death Risk1 = = Risk 0 = = ARR = = NNT = 1 / = 1670

49 NNH number needed to harm Chỉ Fêu E+P (n = 5964) Placebo (n = 5566) Invasive breast cancer Stroke 16 8 Hip fracture 5 8 Death Risk1 = = Risk 0 = = ARR = = NNH = 1 / = 286

50 NNT is dependent on baseline risk Trial Agent Risk profile Placebo Active NNT VERTEBRAL FRACTURE FIT-I Alendronate Prev fx, T < PROOF Calcitonin Prev fx, T < MORE-2 Raloxifene Prev fx, T < VERT-US Risedronate Prev fx, T < VERT-MN Risedronate Prev fx, T < Neer, 2001 PTH 20 mg Prev fx, T < Neer, 2001 PTH 40 mg Prev fx, T < FIT-2 Alendronate No prev fx Alendronate No prev fx, T< MORE Raloxifene 60 No prev fx Raloxifene120 No prev fx TROPOS Strontium T< Strontium Strontium Prev fx HORIZON Zoledronate Prev fx

51 NNT và baseline risk Number needed to treat (NNT) Incidence of vertebral fracture in placebo group

52 Risk communicafon

53 The effect of risk framing (communication) Which (hypothefcal) study is more likely to be funded? T Fahey, et al. Evidence based purchasing: understanding results of clinical trials and systema:c reviews. BMJ 1995;311: Funding score (Max = 100) Mammography Cardiac Rehab RR ARR %Event-free NNT

54 Prescription of drug Questionnaire A on relative risk reduction 1. A cholesterol lowering drug treatment reduces the relative risk of a fatal and non-fatal myocardial infarction by 34%. This result is significant A cholesterol lowering drug treatment reduces the relative risk of a fatal myocardial infarction by 26%. This result is not significant A cholesterol lowering drug treatment increases the total mortality by 5.7%. This result is not significant During a cholesterol lowering drug treatment, 71 patients have to be treated for five years to prevent one fatal or non-fatal myocardial infarction. This result is significant. Questionnaire B on absolute risk reduction 1. A cholesterol lowering drug treatment reduces the incidence of fatal and non-fatal myocardial infarction by 14 per 1000 patients and five years of treatment. This result is significant A cholesterol lowering drug treatment decreases the incidence of fatal myocardial infarction by one per 1000 patients and five years of treatment. This result is not significant A cholesterol lowering drug treatment increases the total mortality by 1.2 deaths per 1000 patients and five years of treatment. This result is not significant During a cholesterol lowering drug treatment 71 patients have to be treated for five years to prevent one fatal or non-fatal myocardial infarction. This result is significant.

55 RR result and prescription of drug Score of treatment initiation Relative risk Myocardial infarction Absolute risk

56 Summary RR is a common effect size measure OR is an es)mate of rela)ve risk For diseases with high risk, OR tends to overes)mate effect size. Presenta)on of RR can influence decision- making NNT is probably more useful than RR/OR, but watch out for background risk

57 "Medicine is a science of uncertainty and an art of probability" (William Osler) There will always be a need for art of medicine to interpret risk predic)ons and use them to inform pa)ent decisions