Evaluating Effectiveness of Treatments. Elenore Judy B. Uy, M.D.

Transcription

1 Evaluating Effectiveness of Treatments Elenore Judy B. Uy, M.D.

2 OUTLINE 1. Why evaluate effectiveness 2. How to acquire evidence 3. How to appraise evidence Directness Validity Results 4. Our Case

3 WALA WALA WALANG HIMALA! x2 24x32 Acrylic on Canvas Geloy Concepcion 2012 Walang himalaaaaaaa!!!

4 1 WHY EVALUATE EVIDENCE

5 Half of what we learn in medical school is wrong 1960 s 1980 s s

6 Clinical Questions Internal Medicine Residents 2 questions for every 3 outpatients 5 for every inpatient 29% pursued books (31%); journals (21%); others (17%) Barriers: lack of time (60%), forgot (29%) Am J Med 2000;109:

7 Millions of Articles ,000 added in 2009 alone Diabetes: 150,000 since 1966 Hypertension: 4000 in October NATIONAL LIBRARY OF MEDICINE

8 The need for a new paradigm We have a continuing need to acquire medical information Knowledge constantly grows and changes We don t have enough time

9 EBM: a new paradigm A systematic approach to ACQUISITION, APPRAISAL, AND APPLICATION of research evidence To guide decisions In healthcare

10 Mrs. Cruz

11 Why evaluate effectiveness Complex medical conditions Limited Resources Each treatment needs to be effective

12 Mrs. Cruz Discharge Diagnosis Community Acquired Pneumonia Pulmonary Tuberculosis (probably inactive) Rheumatic Heart Disease Moderate Mitral Stenosis Transient Ischemic Attack Hypertension Dyslipidemia Medications 1. Azithromycin 500mg OD 2. Cefuroxime 500mg OD 3. Quadruple anti-koch s OD 4. Losartan 50mg OD 5. Metoprolol 50mg OD 6. Spironolactone 25 mg OD 7. Digoxin 0.25 mg OD 8. Warfarin 2.5 mg OD 9. Simvastatin 20mg HS 10. Multivitamins OD 11. Vitamin E OD

13 2 ACQUIRING THE EVIDENCE

14 Forming answerable clinical questions Patient Among adults Exposure is giving multivitamins Outcome beneficial? Morbidity? Mortality?

15 Searching for evidence

16 Gaziano J, Sesso HD. Multivitamins in the Prevention of Cancer in Men: The Physicians Health Study Ii Randomized Controlled Trial. JAMA 308, no. 18 (November 14, 2012): 1871 Acquisition of Evidence doi: /jama Gaziano J, Sesso HD. Multivitamins in the Prevention of Cancer in Men: The Physicians Health Study Ii Randomized Controlled Trial. JAMA 308, no. 18 (November 14, 2012): doi: /jama

17 Directness Validity Results 3 APPRAISING THE EVIDENCE

18 APPRAISING DIRECTNESS Is this study likely to answer my clinical question?

19 Directness Clinical Question Question Research Question (Trial) Population Adult, female Adult, male Exposure Outcome Multivitamins Morbidity Mortality Multivitamins vs. Placebo Cancer

20 APPRAISING DIRECTNESS Is this study likely to answer my clinical question?

21 APPRAISING VALIDITY Is the difference in outcome events really because of the therapies that are being compared? How fair are the comparisons between the therapies being compared? Were the odds stacked in favor of one group over the other (bias)?

22 APPRAISING VALIDITY 1. Randomization 2. Allocation Concealment 3. Equal at baseline 4. Patient blinding 5. Caregiver blinding 6. Outcome assessor blinding 7. Analyzed to original group 8. Adequate follow-up rate

23 1 Randomization Look for: Coin Toss Table of random numbers Computer-generated random sequences

24 2 Allocation Concealment Look for: Sealed, opaque envelopes Third party allocates treatment Computerized allocation of treatment

25 3 Groups equal at baseline Look for the table of baseline characteristics (Table 1)

26 4-6 Blinding Make unaware of treatment assignment Who is blinded? Patient Caregiver Outcome Assessor

27 7 Analyzed to original group Look for: Patients analyzed to the groups to which they were originally randomized (regardless of compliance to treatment) Intention-to-treat analysis done Number of patients randomized=number of patients analyzed

28 8 Adequate follow-up Drop-outs Patients who leave the study Usually leave due to adverse events or dissatisfaction Outcomes unknown

29 8 Adequate follow-up Is the drop-out rate worrisome? Control Drop-out Rate Treatment Drop-out Rate Control Death Rate Treatment Death Rate Reported 1% 1% 2% 1% Worst Case 1% 1% 2% 2%

30 8 Adequate follow-up When to worry: 1. When there is gross imbalance in drop-out rates between groups 2. When drop-out rates are greater than event rates 3. When worst assumptions on what happened lead to opposite conclusions.

31 Appraisal: Validity Validity Criteria Randomly assigned to treatment groups? Allocation concealed? Baseline characteristics similar at the start of the trial? Patients blinded to treatment assignment? Caregivers blinded to treatment assignment? Outcome assessors blinded to treatment assignment? All patients analyzed in the groups to which they were originally randomized? Follow-up rate adequate? Assessment Yes Yes Yes Yes Yes Yes Yes No Best Case (Assume drop outs in treatment survived): 0.92 Worst Case (Assume drop outs in treatment died): 0.99

32 APPRAISING VALIDITY Is the difference in outcome events really because of the therapies that are being compared?

33 APPRAISING RESULTS Am I confident that treatment is effective?

34 APPRAISING RESULTS 1. How large was the treatment effect?

35 My sister is not a pig. She weighs 75% of what she used to weigh 60kg/80kg Relative Weight She lost25% of her weight (80kg- 60kg)/80kg Relative Weight Reduction She lost 20 kilograms 80kg 60kg - Absolute Weight Reduction

36 Risk The probability that an event will occur. -Carr, S. et. al. Developed Outcome Total Patients Treatment Control The risk of developing the outcome: Control (Rc) or Baseline risk : Treatment (Rt) : 8/100 6/100

37 Expressing the change in risk Comparing the risk of treatment (Rt) against the risk of control (Rc): Developed Outcome Total Patients Risk Treatment % Control % My risk is 75% of what it used to be 6 8 Rt Rc Relative Risk I lost 25% of my risk (8-6) 8 (Rc-Rt) Rc Relative Risk Reduction I lost 2% of my risk 8-6 Rc-Rt Absolute Risk Reduction

38 1 How large was the treatment effect My risk is now 75% of what it used to be Relative Risk Rt/Rc I lost 25% of my risk Relative Risk Reduction (Rc-Rt)/Rc I lost 2% of my risk Absolute Risk Reduction Rc-Rt

39 Relative Risk (RR) RR = Risk of Treatment (Rt) Risk of Control (Rc) Assuming a NEGATIVE outcome/event: RR Value Implication Interpretation RR>1 Rt>Rc Harm RR=1 Rt=Rc No effect RR<1 Rt<Rc Benefit

40 Point Estimates RR, RRR, ARR Best estimate of the true effect of a drug Misleading Does not accurately reflect the uncertainty surrounding the estimate due to sample size limitations Among adults who take multivitamins, the risk of developing cancer is decreased to 92% of the risk of those who do NOT take it

41 Interval Estimates Confidence Intervals Humbler, admits a range of possible values of the treatment effect Are more likely to be correct Provides best and worse case scenario Among adults who take multivitamins, the risk of developing cancer is decreased to 92% of the risk of those who do NOT take it [95% CI: 0.86, 0.998]

42 APPRAISING RESULTS 1. How large was the treatment effect? 2. How precise was the estimate of effect?

43 Precision

44 Interval Estimates Benefit 1 Harm

45 Appraisal: Results Incidence of total cancers: HR: 0.92 (95%CI: 0.86 to 0.998) 1. How large was the treatment effect? Among adults who take multivitamins, the risk of developing cancer is decreased to 92% of the risk of those who do NOT take it. 2. How precise was the estimate of the treatment effect? At best: the risk of developing cancer in those who take multivitamins is 86% the risk of those who do NOT take it. At worst: the risk of developing cancer is practically the same in those who do and do NOT take multivitamins (99.8% 100%)

46 APPRAISING RESULTS Am I confident that treatment is effective?

47 Recommendation: DO NOT recommend taking multivitamins It may offer no benefit over placebo HR: 0.92 (95%CI: 0.86 to 0.998)

48 4 OUR CASE: Mrs. Cruz

49 Mrs. Cruz Discharge Diagnosis Community Acquired Pneumonia Pulmonary Tuberculosis (probably inactive) Rheumatic Heart Disease Moderate Mitral Stenosis Transient Ischemic Attack Hypertension Dyslipidemia Medications 1. Azithromycin 500mg OD 2. Cefuroxime 500mg OD 3. Quadruple anti-koch s OD 4. Losartan 50mg OD 5. Metoprolol 50mg OD 6. Spironolactone 25 mg OD 7. Digoxin 0.25 mg OD 8. Warfarin 2.5 mg OD 9. Simvastatin 20mg HS 10. Multivitamins OD 11. Vitamin E OD

50 Mrs. Cruz Discharge Diagnosis Community Acquired Pneumonia-resolved Pulmonary Tuberculosis (probably inactive) Rheumatic Heart Disease Moderate Mitral Stenosis Transient Ischemic Attack Hypertension Dyslipidemia Medications 1. Metoprolol 50mg OD 2. Spironolactone 25 mg OD 3. Warfarin 2.5 mg OD 4. Simvastatin 20mg HS

51 OUTLINE 1. Why evaluate effectiveness 2. How to acquire evidence 3. How to appraise evidence Directness Validity Results 4. Our Case

52

53

54 Walang himala! Ang himala ay nasa puso ng tao, nasa puso nating lahat! Tayo ang gumagawa ng mga himala!

55 Thank you.

56 Relative Risk Reduction (RRR) RRR = Risk of Control (Rc) - Risk of Treatment (Rt) Risk of Control (Rc) Assuming a NEGATIVE outcome/event: RR Value Implication Interpretation RRR>0% Rt<Rc Benefit RRR=0% Rt=Rc No effect RRR<0% Rt>Rc Harm

57 Absolute Risk Reduction (ARR) ARR = Risk in Control (Rc)-Risk in Treatment (Rt) Assuming a NEGATIVE outcome/event: RR Value Implication Interpretation ARR>0% ARR=0% ARR<0% Rt<Rc Rt=Rc Rt>Rc Benefit No effect Harm

58 8 Adequate follow-up Is the drop-out rate worrisome? Control Drop-out Rate Treatment Drop-out Rate Control Death Rate Treatment Death Rate Reported 1% 10% 5% 5% Worst Case 1% 10% 5% 15%

59 8 Adequate follow-up Is the drop-out rate worrisome? Control Drop-out Rate Treatment Drop-out Rate Control Death Rate Treatment Death Rate Reported 10% 10% 10% 5% Worst Case 10% 10% 10% 15%