Indonesia Clinical epidemiology and Evidence Based Medicine (ICE(ICE-EBM) HARM Jarir At Thobari (FK UGM)
Efficacious Medical intervention Harm
HARM Unintended physical injury resulting from or contributed to by medical care, that requires additional monitoring, treatment or hospitalization, or that results in death which is hospital acquired. Definition modified from the IHI definition of Harm by the QUEST Harm Workgroup
Harm Do oral contraceptives cause breast cancer? Do calcium antagonists increase the risk of death or cancer? Do vasectomies increase the risk of prostate cancer? Does Rosiglitazone cause myocardial infaction?
Summary estimates of risk of breast cancer in premenopausal women and women younger than 50 years associated with ever use of oral contraceptives Kahlenborn C et al. Mayo Clin Proc. 2006;81:1290-1302
Risk of breast cancer in premenopausal women and women younger than 50 years associated with ever use of oral contraceptives Meta analysis OR=1.19; 95% CI, 1.09-1.29 Parous OR=1.29; 95% CI: 1.20-1.40 Nulliparous OR=1.24; 95% CI: 0.92-1.67 Parous women, OCs were used Before FFTP* OR=1.44 (1.28-1.62) after FFTP OR=1.15 (1.06-1.26) used OCs 4 or more years before FFTP (OR=1.52 (1.26-1.82). *first full-term pregnancy Mayo Clin Proc. 2006;81(10):1290-1302
Primary and Other Outcomes: DREAM Rosiglitazone group (n=2635) Placebo group (n=2634) HR (95% CI) p Cardiovascular events composite* 75 (2.9%) 55 (2.1%) 1.37 (0.97-1.94) 0.08 Myocardial infarction 15 (0.6%) 9 (0.3%) 1.66 (0.73-3.80) 0.2 Stroke 7 (0.3%) 5 (0.2%) 1.39 (0.44-4.40) 0.6 Cardiovascular death 12 (0.5%) 10 (0.4%) 1.20 (0.52-2.77) 0.7 Confirmed heart failure 14 (0.5%) 2 (0.1%) 7.03 (1.60-30.9) 0.01 New angina 24 (0.9%) 20 (0.8%) 1.20 (0.66-2.17) 0.5 Revascularization 35 (1.3%) 27 (1.0%) 1.29 (0.78-2.14) 0.3 Myocardial infarction, stroke, or cardiovascular death 32 (1.2%) 23 (0.9%) 1.39 (0.81-2.37) 0.2 Data are number (%). *Rows are not mutually exclusive for components of the composite if a participant had more than one component of the composite then they are counted in the relevant row. Regression implies achieving a normal fasting glucose concentration (as defined in both rows) and 2-h plasma glucose level. Defined as acute treatment with at least two of the following criteria: typical signs and symptoms, typical radiological evidence, use of diuretics, vasodilators, or inotropes. DREAM investigators. Lancet 2006;368:1096-1105.
Adverse Events, Hospitalization, and Death: ADOPT Variable Adverse events number of patients (%) Rosiglitazone (N=1456) Metformin (N=1454) Glyburide (N=1441) Serious Events Total Events Serious Events Total Events Serious Events Total Events Total events 346 (23.8) 1338 (91.9) 331 (22.8) 1341 (92.2) 308 (21.4) 1321 (91.7) Cardiovascular disease 49 (3.4) 62 (4.3) 46 (3.2) 58 (4.0) 26 (1.8) 41 (2.8) Myocardial Infarction Fatal 2 (0.1) 2 (0.1) 2 (0.1) 2 (0.1) 3 (0.2) 3 (0.2) Nonfatal 22 (1.5) 25 (1.7) 18 (1.2) 21 (1.4) 11 (0.8) 15 (1.0) Congestive heart failure (investigator-reported) 12 (0.8) 22 (1.5) 12 (0.8) 19 (1.3) 3 (0.2) 9 (0.6) Stroke 13 (0.9) 16 (1.1) 17 (1.2) 19 (1.3) 12 (0.8) 17 (1.2) Peripheral vascular disease 7 (0.5) 36 (2.5) 6 (0.4) 27 (1.9) 4 (0.3) 31 (2.2) Gastrointestinal events 8 (0.5) 335 (23.0) 7 (0.5) 557 (38.3) 3 (0.2) 316 (21.9) Death and Hospitalization Hospitalization for any reason Patients no. (%) 169 (11.6) 172 (11.8) 150 (10.4) Events no. 251 267 203 Deaths from any cause no. 34 31 31 P<0.05 for the comparison between this treatment group and the rosiglitazone group; P<0.01 for the comparison between this treatment group and the rosiglitazone group. ADOPT study group. N Engl J Med 2006;355:2427-2443.
HOME SUBSCRIBE CURRENT ISSUE PAST ISSUES COLLECTIONS HELP Search Term Advanced Search S A correction Search E A R C HNEJM has been published: N Engl J Med 2007;357(1):100. Institution: Syrian Arab Republic Sign In as Individual Contact Subscription Administrator at Your Institution FAQ Volume 356:2457-2471 June 14, 2007 Number 24 Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes Steven E. Nissen, M.D., and Kathy Wolski, M.P.H
Meta Analysis: Rates of Myocardial Infarction and Death from Cardiovascular Causes Rosiglitazone Group Control Group Odds Ratio (95% CI) P Value no. of events/total no. (%) Myocardial Infarction Small trials combined 44/10,285 (0.43) 22/6106 (0.36) 1.45 (0.88 2.39) 0.15 DREAM 15/2,635 (0.57) 9/2634 (0.34) 1.65 (0.74 3.68) 0.22 ADOPT 27/1,456 (1.85) 41/2895 (1.42) 1.33 (0.80 2.21) 0.27 Overall 1.43 (1.03 1.98) 0.03 Death from Cardiovascular Causes Small trials combined 25/6,845 (0.36) 7/3980 (0.18) 2.40 (1.17 4.91) 0.02 DREAM 12/2,635 (0.46) 10/2634 (0.38) 1.20 (0.52 2.78) 0.67 ADOPT 2/1,456 (0.14) 5/2895 (0.17) 0.80 (0.17 3.86) 0.78 Overall 1.64 (0.98 2.74) 0.06 Nissen and Wolski. N Engl J Med 2007;356:2457-2471.
Hierarchy of Evidence Systematic Reviews of Randomized Controlled Trials (Meta-analysis) Single Randomized Controlled Trial (RCT) Systematic Review of Observational Studies Addressing Patient-Important Outcomes Single Observational Study Addressing Patient-Important Outcomes Physiologic Studies Unsystematic Clinical Observations
Level 1a 1b Therapy/Prevention, Aetiology/Harm SR (with homogeneity*) of RCTs Individual RCT (with narrow Confidence Interval ) Prognosis Diagnosis Differential diagnosis/symptom prevalence study SR (with homogeneity*) of inception cohort studies; CDR validated in different populations Individual inception cohort study with > 80% follow-up; CDR validated in a single population SR (with homogeneity*) of Level 1 diagnostic studies; CDR with 1b studies from different clinical centres Validating** cohort study with good reference standards; or CDR tested within one clinical centre SR (with homogeneity*) of prospective cohort studies Prospective cohort study with good follow-up**** Economic and decision analyses SR (with homogeneity*) of Level 1 economic studies Analysis based on clinically sensible costs or alternatives; systematic review(s) of the evidence; and including multi-way sensitivity analyses 1c All or none All or none case-series Absolute SpPins and SnNouts All or none case-series Absolute better-value or worse-value analyses 2a 2b 2c SR (with homogeneity*) of cohort studies Individual cohort study (including low quality RCT; e.g., <80% follow-up) "Outcomes" Research; Ecological studies SR (with homogeneity*) of either retrospective cohort studies or untreated control groups in RCTs Retrospective cohort study or follow-up of untreated control patients in an RCT; Derivation of CDR or validated on splitsample only SR (with homogeneity*) of Level >2 diagnostic studies Exploratory** cohort study with good reference standards; CDR after derivation, or validated only on split-sample or databases SR (with homogeneity*) of 2b and better studies Retrospective cohort study, or poor follow-up SR (with homogeneity*) of Level >2 economic studies Analysis based on clinically sensible costs or alternatives; limited review(s) of the evidence, or single studies; and including multi-way sensitivity analyses "Outcomes" Research Ecological studies Audit or outcomes research 3a SR (with homogeneity*) of case-control studies SR (with homogeneity*) of 3b and better studies SR (with homogeneity*) of 3b and better studies SR (with homogeneity*) of 3b and better studies 3b Individual Case-Control Study Non-consecutive study; or without consistently applied reference standards Non-consecutive cohort study, or very limited population Analysis based on limited alternatives or costs, poor quality estimates of data, but including sensitivity analyses incorporating clinically sensible variations. 4 Case-series (and poor quality cohort and casecontrol studies ) Case-series (and poor quality prognostic cohort studies***) Case-control study, poor or nonindependent reference standard Case-series or superseded reference standards Analysis with no sensitivity analysis 5 Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on economic theory or "first principles"
Clinical Scenario Osteoarthritis Osteoarthritis is the most common type of arthritis, especially among older people People with osteoarthritis usually have joint pain and limited movement Unlike some other forms of arthritis, osteoarthritis only affects joints, and not internal organs Usually need long-term analgesics Is there any side effects of using long-term analgesics?
Clinical Question P I C O Patient and the problem: osteoarthritis Intervention: selective Cox-2 inhibitor (rofecoxib) Comparison: other analgesic and antiinflammatory agents Outcome of interest: congestive heart failure
BMJ 2005; 330:1370
How to assess HARM? Is there evidence on cause & effect relationship? Valid Importance Direct relevance
Establishing Validity
1. Were there clearly defined group of patients, similar in all important ways other than exposure to the treatment or other cause? Randomization?? The choice of comparison groups has an enormous influence on the credibility of the results The design of the study determines the comparison groups
Randomized Controlled Trials (RCT) Not suitable for harm studies: Unethical Expensive (Rare events, long period of time) Need large samples Some RCTs for therapy might have shown harm unintentionally (stopped)
Cohort Studies Prospective, Non-randomized exposure Useful when subjects cannot be assigned to an exposure group Exposed patients might differ from non-exposed for important determinants of outcome (CONFOUNDERS) Investigator must document the characteristics of each group and demonstrate comparability, and use statistical techniques to adjust for known confounders
Confounders (Atthobari et al, Nephrol. Dial. Transp 2006; 21:3106-14)
Case-Control Useful for RARE outcomes that take LONG time to develop Ex: DES and vaginal adenocarcinoma in female Susceptible to unmeasured confounders Retrospective Recall Bias
Cross Sectional Studies Can be used to study potentially causal relationships between risk factors & outcomes Measure prevalence (not incidence) Exposures (risk factors) and outcomes (disease) are measured simultaneously Temporal relationship is lost
Case Series and Case Reports No comparison group! Unusual/dramatic outcome (Phocomelia in offsprings of mothers receiving Thalidomide) Sufficient for hypothesis generation (Need more studies)
Design Starting Point Assessment Strengths Weaknesses Cohort Exposure status Outcome event status Feasible when impossible to randomize -Susceptible to bias -Limited validity Case- Control Outcome event status Exposure status -Overcomes temporal delays -May only requires small sample size -Susceptible to bias -Limited validity RCT Exposure status Adverse event status Low susceptibility to bias -Feasibility -Generalizability
2. Were treatment/exposures & clinical outcomes measured in the same ways in both groups? (Was the assessment of outcomes either objective or blinded) How was the exposure ascertained? Recall Bias or Interviewer Bias Blinding How was the outcome ascertained? Surveillance Bias Blind Study hypothesis
3. Was the follow up of the study patients sufficiently long (for the outcome to occur) and complete? Short term Follow up 20% lost to follow up Long term celecoxib, rofecoxib, NSAID CHF No CHF
4. Do the results satisfy some diagnostic tests for causation? CAUSE Temporality Strength of association Biologic Plaucibility Specificity Dose-response Consistency EFFECT Cessation of exposure
ARE THE VALID RESULTS OF THIS HARM STUDY IMPORTANT? If the study is valid in showing harm/association, how bad and how accurate is the harm?. In other words What is the magnitude and precision of the association between exposure and outcome?
The Estimate of the Magnitude Depends on the Study Design
RCT Cohort Incidence or risk Harm No I exp = 20/1000 = 0.02 I Treatment 20 1000 non-exp = 2/1000 = 0.002 No treatment 2 1000 RR= I exp / I non non-exp exp= 10 Exposed patients are 10 times more likely to suffer from the adverse event than non-exposed
ADVERSE OUTCOME CHF No CHF Celecoxib a b EXPOSE TO TREATMENT NSAID c d RR = a / (a+b) ---------- c/ (c+d) OR = ad /bc
Case-Control Studies Odds Ratio (Relative Odds) Adverse Outcome Present Absent Totals Exposure a b a+b Yes Exposure No Totals c d c+d a+c b+d OR = (a/c)/(b/d) = ad/bc a+b+c+ d
Are the Results Important? Case-Control Studies Out of 100 affected cases 90 were exposed to the agent (a) 10 were not exposed (c) Out of 100 non-affected cases (controls) 45 were exposed to the agent (b) 55 were not exposed (d)
Case Control Studies Odds Ratio Out of 100 affected cases 90 were exposed 10 were not exposed Out of 100 controls 45 were exposed 55 were not exposed Exposure Yes Exposure No Adverse Outcome Present Absent Totals 90 45 135 10 55 65 Totals 100 100 200 Odds Ratio: OR = (a/c)/(b/d) = ad/bc = (90x55)/(45x10) = 11 The odds of experiencing the adverse event for patients exposed is 11x that of those unexposed
Odds Ratio (OR) OR > 1 represents an increased risk or association Describes the relative harm of an exposure independent of disease prevalence When the prevalence of the outcome of interest is rare in the population from which the sample was drawn (often the reason for using a casecontrol study), the OR closely approximates the RR
Criteria to Examine RR & OR Magnitude Cohort and Case-Control are both biased What if RR or OR s value preferable? R.R. > 3 O.R. > 4 Impressive!! Recalculate adjusting for a known confounder, if increased or the same after adjustment more believable If the adjustment results in large decline in RR or OR We should be suspicious of both of them Precision
Adjusting for Confounders Assuming the study is attempting to examine the association between SSRIs and Upper GI bleed. 100 cases with GI bleed, 90 were on SSRIs, 100 controls with no GI bleed, 45 were on SSRIs OR = 11 If in the study 7 subjects were also on Ibuproffen (confounder) (2 in the control group one on SSRI, 5 in the cases group 3 on SSRI) You can calculate an adjusted OR
Adjusted OR Upper GI Bleed 7 subjects on Ibuproffen 2 in the control group one on SSRI (b) 5 in the cases group 3 on SSRI (a) Exposure Yes Present Absent Totals 90 45 135 87 44 131 Exposure 10 55 65 No 8 54 62 Totals 100 100 200 95 98 193 The adjusted OR = ad/bc = (87x54)/(44x8) = 13 What is your conclusion????
Criteria to Examine RR and OR Magnitude Precision Is highest when C.I. given is narrow and remains within a clinically important increased risk
Examples of C.I. Is There Association?? SSRIs and Upper GI bleed: Adjusted OR 3.0 (95% CI, 2.1-4.4) Non-SSRIs and Upper GI bleed: Adjusted OR 1.4 (95% CI, 1.1-1.9) NSAIDs and Upper GI bleed: Adjusted OR 3.7 (95% CI, 3.2-4.4) SSRIs + NSAIDs and Upper GI bleed: Adjusted OR 15.6 (95% CI 6.6-36.6) Long acting Benzodiazepines and risk of fall in elderly patients: RR 2.0 (95% CI 1.6-2.5) Calcium channel blockers and MI: OR 1.9 (95% CI 0.9-5.5)
Number Needed to Harm (NNH) The number of patients who, if they received the experimental treatment, would lead to one additional patient being harmed, compared with patients who received the comparison treatment. 1/ARI
NNH For RCT and Cohort One way to report the evidence to your patients Same formula as NNT NNH = 1/ARI = 1/ (I exp - I nonexp ) = 1 [a/(a+b)-c/(c+d)] ARI = Absolute Risk Increase
Number Needed to Harm (NNH) NNH = a a + b 1 c c + d If OR < 1 If OR > 1 1 - [PEER x (1 - OR)] NNH = ----------------------------- (1 - PEER) x PEER x (1 - OR) 1 + [PEER x (OR - 1)] NNH = ----------------------------- (1 - PEER) x PEER x (OR - 1)
NNH - For Case Control PEER = Patient Expected Event Rate = Adverse event rate in non exposed Table of values available (Do NOT Need to Memorize) Note: For the same OR, NNH can be different for different PEERs: Example: For OR = 0.9 NNH = 2000 for PEER of 0.005 NNH = 40 for PEER of 0.40
Converting OR to NNH For OR greater than 1 1.1 1.25 1.5 1.75 2 2.25 2.5 PEER 0.05 212 86 44 30 23 18 16 0.1 113 46 24 16 13 10 9 0.2 64 27 14 10 8 7 6 0.3 50 21 11 8 7 6 5 0.4 44 19 10 8 6 5 5 0.5 42 18 10 8 6 6 5 0.7 51 23 13 10 9 8 7 0.9 121 55 33 25 22 19 18 Calculator available at: http://www.cebm.utoronto.ca/practise/ca/statscal/ortonnt.htm
The Flawless Study Flawful A young investigator is designing a case-control study on the association between SSRIs and upper GI bleed. He recruited his cases and controls from a GI clinic. Both cases and controls were interviewed retrospectively for their use of SSRIs. In order to blind himself to the subjects response, interviews were conducted by the clinic nurse who would ask the subjects a set of questions while checking them in for the visit. The investigator reported a RR of 2.5 (95% CI 0.5-4.0) and concluded that there is a positive association between SSRIs and upper GI bleed and that 1 in 2-3 patients treated with SSRIs will have an upper GI bleed caused by the medication
Study Flaws Controls might have other GI diseases that predisposes them to GI bleed Interviewer not blinded CI wide and lower end < 1 Used RR in a case-control design Interpretation of RR is wrong
Summary Harm is unavoidable but can be minimized Every intervention has some degree of harm Criteria for establishing validity in a harm study RCT, while excellent design, might be unethical in harm Cohort/ Case-Control (Rare outcome or outcome that takes long time to develop) Association versus Causation
Summary Magnitude: RCT or Cohort RR = I exp /I nonexp = [a/(a+b)]/[c/(c+d)] Case-Control OR = (a/c)/(b/d) = ad/bc Magnitude: RR or OR > 1 (positive association) Preferable association: (because of confounders) RR > 3 OR > 4 More criteria needed for CAUSATION Adjustment for confounders Precision: C.I. (Narrow and do not straddle 1 ) NNH = 1/ARI = 1/ (I exp - I nonexp )
THANK YOU