Quantifying the Benefits and Harms of Interventions: Relative vs. Absolute Risk

JOURNAL CLUB LECTURE SERIES: Quantifying the Benefits and Harms of Interventions: Relative vs. Absolute Risk Tuesday, May 12 th, 2015 Anil N. Makam, MD MAS Oanh K. Nguyen, MD MAS Division of General Internal Medicine Division of Outcomes & Health Services Research UT Southwestern Medical Center

Overall Objective of JC Summarize an article for a resident and you ve taught them a new fact. Teach a resident to critically appraise the literature and you taught them to be a life learner! Disclaimer: You will NOT learn these skills by showing up to Journal Club without having read the article in advance. This takes ~60 minutes to do well, but something is better than nothing. Don t let perfect be the enemy of good.

Roadmap (this may change) Overview August: overview of research, funding, RQ, predictors, outcomes Study design series: September: RCT Part 1 October 7th: RCT Part 2 October 21st: RCT Part 3 November: RCT Part 4 December: Observational cohort studies part 1 January: Observational cohort studies part 2 February: Cancelled March: Observational cohort studies part 3 Interpretation of results: April: P-values and confidence intervals May: Absolute (ARR, NNT) vs relative risks (RR, OR)

Anatomy of interpreting a paper Authors & funding sources Research question Study design Study subjects and how they will be sampled Variables and how they will be measured Predictor Outcome Confounders Source of history Chief concern HPI Interpretation of findings and statistical analyses Conclusion: validity, inferences, take-home message Physical Exam/Labs/Imaging Assessment/Plan

Overall Objective To provide you a foundational understanding and skills needed to interpret and apply relative and absolute risk to critically appraise the literature AND to clinical practice (EBM) Please review previous lectures on http://imweb.swmed.edu/

Rationale Any treatment involves tradeoffs Weigh benefits against risks/costs This decision can be difficult How big is this box? And this one? Benefit $$ Harm Treatment

Rationale Effect size estimated in studies can help us figure out the size of the boxes How big is this box? And this one? Benefit $$ Harm Treatment

Rationale: You will encounter this routinely How strongly should you discourage consumption of deli meats and soft cheese to prevent listeriosis? We will come back to this

Lecture Objectives Understand how to determine if an intervention is beneficial using effect size: Relative risk and relative risk reduction Absolute risk & number needed to treat (NNT) Basic calculation examples Examples applying lessons learned to patient care

Is the Intervention Beneficial? Studies compare an outcome in treated versus untreated persons Example: MI occurred in 10% in treated vs. 15% in untreated (P-value=0.03) P-values are used to decide if we should reject the null hypothesis and accept that the intervention is beneficial

Is the Intervention Beneficial? But p-values cannot help us interpret how beneficial (i.e. the effect size) Smaller the p-value does not mean more effective the intervention, and vice versa Statistical significance (p<.05) does not mean clinical significance, and vice versa

Quantifying the Benefit Effect size How do we summarize & communicate this? What is really important for clinicians and policymakers? Example: MI in 10% (tx) vs. 15% (ctrl) How would you summarize the effect size? What do we do with these two numbers? 14

Quantifying the Benefit Two possibilities for effect size: 10% / 15% = 0.66 15% - 10% = 5% Relative Risk (RR) Absolute Risk Reduction (ARR) 15

Quantifying the Benefit Relative risk as a measure of effect size Medium Big Small RR = 0.66 is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime RR is NOT the best measure of effect size

Neither are Odds Ratio Odds ratios are another measure of relative benefit/harm Same limitations for quantifying benefits/harms using RRs apply to ORs RR = events/total at risk OR = events/non-events ORs are always larger than RR, but if event rate<10%, it approximates RR

ORs are always larger than RRs Outcome Rate in Control Group For fixed relative risk, ORs will increase as outcome rate increases

Quantifying the Benefit Absolute risk reduction (ARR) is better ARR = Risk difference = Risk 2 Risk 1 RR ARR MI: 10% vs. 15% in 10 years.66??? Death: 50% vs. 75% in 3 years.66??? Basal Cell CA: 2% vs. 3% in lifetime.66???

Quantifying the Benefit Absolute risk reduction (ARR) is better ARR = Risk difference = Risk 2 Risk 1 RR ARR MI: 10% vs. 15% in 10 years.66 5% Death: 50% vs. 75% in 3 years.66 25% Basal Cell CA: 2% vs. 3% in lifetime.66 1%

What does the 34% difference mean?

Nimotop Ad Graph Risk(tx) = 61/278 = 22% Risk(ctrl) = 92/276 = 33% RR = 22%/33% =.66 ARR = 33% -22% = 11% What is 34%? Relative risk reduction (RRR) = 1 RR = 1-.66 =.34 or 34%

Quantifying the Benefit Relative risk reduction (RRR) is no better than relative risk (RR) RR RRR MI: 10% vs. 15% in 10 years.66 34% Death: 50% vs. 75% in 3 years.66 34% Basal Cell CA: 2% vs. 3% in lifetime.66 34% RRR is ALWAYS bigger than ARR

Quantifying the Benefit BEWARE of risk reduction language!!! Is this ARR or RRR? We reduced risk by 34% Risk was 34% lower can t tell can t tell

Quantifying the Benefit Another reason that ARR is better: Translate into number needed to treat NNT = 1/ARR

Why is NNT = 1/ARR? 100 SAH patients treated 67 no stroke anyway 11 strokes prevented R2 R1 22 strokes with Nimotop ARR = R2 R1 ARR = 33 22 = 11 100 100 100 33 strokes with no treatment 22 strokes with with treatment

Why is NNT = 1/ARR? Treat 100 SAH patients prevent 11 strokes Ratio manipulation: 100 treated 1 treated 9.1 treated = = 11 prevented.11 prevented 1 prevented 1/ARR = NNT

Quantifying the Benefit NNT is best expressed in a sentence: Need to treat 10 persons with SAH using nimodipine for 21 days to prevent 1 cerebral infarction

Lecture Objectives Understand how to determine if an intervention is beneficial using: Relative risk and relative risk reduction Absolute risk & number needed to treat (NNT) Basic calculation examples Examples applying lessons learned to patient care

Example 1 NNT calculation practice: RR ARR NNT MI: 10% vs. 15% in 10 years.66 5%??? Death: 50% vs. 75% in 1 year.66 25%??? Basal Cell CA: 2% vs. 3% in lifetime.66 1%???

Example 1 NNT calculation practice: Statins Chemo Sunscreen every day RR ARR NNT MI: 10% vs. 15% in 10 years.66 5% 20 Death: 50% vs. 75% in 1 year.66 25% 4 Basal Cell CA: 2% vs. 3% in lifetime.66 1% 100 NNT expression practice:

Example 1 NNT expression practice Need to treat 20 patients with statins for 10 years to prevent 1 MI Need to treat 4 patients with chemo for 1 year to prevent 1 death Need to treat 100 patients with sunscreen every day for their whole life to prevent 1 basal cell cancer

Example 2 Warfarin vs. placebo for atrial fibrillation Warfarin Placebo Risk of major bleed (/yr) 1.2% 0.7% What s the RR, ARI (absolute risk increase), and NNH? Ann Intern Med 1999; 131:492-501

Example 2 Warfarin vs. placebo for atrial fibrillation RR = R1/R2 = 1.2% / 0.7% = 1.7 ARI = R1 R2 = 1.2% - 0.7% = 0.5% NNH = 1/ARI = 1/0.5% = 200 If you treat 200 afib patients with warfarin, you will cause 1 major bleed per year Ann Intern Med 1999; 131:492-501

Applying to Individual Patients Example 3: Warfarin vs. placebo for stroke prevention in atrial fibrillation RR = 0.34; RRR = 0.66 How do you estimate how beneficial prescribing warfarin is for an individual patient?

Applying to Individual Patients Example 3: Warfarin vs. placebo for stroke prevention in atrial fibrillation (RR = 0.34; RRR = 0.66) 1. Estimate Risk for Individual: Use CHA2DS-VASc Patient 1: CHA2DS2-VASc = 0 0.7% stroke risk/year without therapy Patient 2: CHA2DS2-VASc = 3 4.3% stroke risk/year without therapy Patient 3: CHA2DS2-VASc = 6 12.5% stroke risk/year without therapy

Applying to Individual Patients Example 3: Warfarin vs. placebo for stroke prevention in atrial fibrillation (RR = 0.34; RRR = 0.66) 2. Calculate NNT for an Individual Patient 1: CHA2DS2-VASc = 0 0.7% stroke risk/year without therapy ARR = 0.7% - (0.7%*.34) = 0.5%; NNT = 1/.005 = 200 Patient 2: CHA2DS2-VASc = 3 4.3% stroke risk/year without therapy ARR = 4.3% - (4.3%*.34) = 2.8%; NNT = 1/0.028 = 35 Patient 3: CHA2DS2-VASc = 6 12.5% stroke risk/year without therapy ARR =12.5% - (12.5%*.34) = 8.3%; NNT=1/0.083= 12

Applying to Individual Patients Example 3: NNT expression practice: Pt 1 with score of 0: NNT = 200 Need to treat 200 patients with CHA2DS2-VASc of 0 with warfarin each year to prevent 1 stroke (but 199 patients derive no benefit) Pt 2 with score of 3: NNT = 35 Need to treat 35 patients with CHA2DS2-VASc of 3 with warfarin each year to prevent 1 stroke (but 34 patients derive no benefit) Pt 3 with score of 6: NNT = 12 Need to treat 12 patients with CHA2DS2-VASc of 6 with warfarin each year to prevent 1 stroke (but 11 patients will derive no benefit)

Applying to Individual Patients What else do you need to know to decide whether to prescribe warfarin? Harms (NNH for bleeding) and costs (drug is cheap but monitoring is cumbersome) Value of stroke compared to major bleed and costs?

Applying to Individual Patients What s the value of the benefits and risks? How bad is 1 stroke compared to 1 bleed? Stroke prevention $$ Bleed Warfarin

Applying to Individual Patients What else do you need to know to decide whether to prescribe warfarin? Harms (NNH for bleeding) and costs (drug is cheap but monitoring is time intensive) Value of stroke compared to major bleed and costs? Decision analyses weight benefits and harms (Ex#4) Guidelines help to determine population thresholds Shared decision making to tailor based on individual tolerance of risks and benefits Best approach is to use pictogram decision aids

Pictogram showing the annual stroke risk of a patient with atrial fibrillation and a CHA2DS2-VASc score of 3. Luke Seaburg et al. Circulation. 2014;129:704-710 Copyright American Heart Association, Inc. All rights reserved.

Applying to Individual Patients: Example 4: Treating Diabetes in the elderly Relative risk reduction for treating diabetes: Visual loss: RRR = 29% per 0.9% in A1c Neuropathy: RRR = 19% per 0.9% in A1c Microalbuminuria: RRR = 33% per 0.9% in A1c UKPDS Group. Lancet 1998

Applying to Individual Patients: Treatment Scenario 1: Patient has A1c=8.5%, you prescribe metformin, A1c is reduced to 7.0% NNT 16 24 48 143 NNT 48 63 100 200 NNT 38 46 67 125 Vijan, S. JAMA Int Med. 2014

Applying to Individual Patients: Treatment Scenario 2: Same patient as before, a1c to 8.5% after 10 years of metformin therapy, you prescribe insulin, a1c back to 7% NNT 77 143 334 1000 NNT 250 500 1000 NNT 250 334 500 1000 Vijan, S. JAMA Int Med. 2014

Applying to Individual Patients: Thou Shall Treat Level of Risk (AND not level of risk factor) Now when you consider disutility of taking metformin/insulin daily and harms of therapy:

Summary Points RR and p-values good for hypothesis testing and statistics (i.e., does it work?) Beware of ambiguous language w/ RR and RRR ARR and NNT (ARI and NNH) much better for interpreting clinical importance (i.e. should you prescribe it? Is it worth it?) ARR = risk difference between groups; NNT = 1/ARR Can be calculated at population or individual level

Take Home Points Thou shall treat level of risk, not level of risk factor SHALL NOT TREAT : SHALL TREAT BASED ON: LDL CV Risk, life expectancy A1c Microvascular risk, life expectancy BMD Prior fracture, FRAX score, life expectancy SBP/DBP CV Risk, life expectancy

Take Home Points Thou shall treat level of risk, not level of risk factor (LDL, A1c, BMD, SBP/DBP) Incorporate your patient s preferences into decision not everyone has same tolerance of risk and benefits HAVE THERAPEUTIC HUMILITY! For most interventions, over 90% (NNT 10), if not over 99% (NNT 100), of your patients will derive zero benefit (aside from placebo effect)

Risk of Listeriosis in Pregnancy 1 case in 83,000 servings of deli meat 1 case in 5 million servings of soft cheese Eat at your own risk Einarson, Can Fam Physician 2010

Acknowledgements Several slides borrowed from: Mark Pletcher, MD, MPH Associate Professor Departments of Epidemiology & Biostatistics and Medicine UCSF

QUESTIONS? OanhK.Nguyen@UTSouthwestern.edu @OanhKieuNguyen Anil.Makam@UTSouthwestern.edu @AnilMakam

Not sure Next Lecture