Right Answers, Wrong Ques2ons Ralph I Horwitz
Disclosures Employed by GlaxoSmithKline Views expressed reflect mine alone and not those of GSK
Right Answer, Wrong Ques2on A young couple moves into an apartment and decides to repaper the dining room. They ask the neighbor who has a dining room the same size, How many rolls of wallpaper did you buy when you papered your dining room? Seven, he says So the couple buys seven rolls of expensive paper, and they start papering. When they get to the end of the fourth roll, the dining room is finished. Annoyed, they go back to the neighbor and say We followed your advice, but we ended up with three extra rolls! So, he says, that happened to you too. (Cathcart and Klein. Plato and a Platypus Walk into a Bar...)
Right Answers, Wrong Ques2ons Emphasis on internal validity has made the quality of the answer more important than quality of ques2on When efficacy trials dominate a collateral effect is to diminish the emphasis on the pa2ent s experience with disease and treatment that are oven bewer captured in effec2veness studies
Right Answers, Wrong Ques2ons: Key Messages Benefits and harms may differ in studies designed for effec2veness rather than efficacy Medicines that have less efficacy in tradi2onal RCTs may be more effec2ve in real- world use Pa2ent- centered informa2on (symptoms, func2on, well- being) needs greater emphasis Observa2onal studies are a model for design and analysis of Effec2veness RCTs
Right Answers, Right Ques2ons For RCTs For Observa2onal studies For Pa2ents
Right Ques2ons for RCTs: Montelukast and Asthma Leukotriene receptor antagonist Approved to treat Asthma Popular with pa2ents and physicians
Efficacy Trials: Montelukast vs ICS and LABA ICS and LABA have greater improvement in lung func2on (FEV1) and airway response to inhaled methacholine, and Greater reduc2on in severe exacerba2ons
From: Oral Montelukast, Inhaled Beclomethasone, and Placebo for Chronic Asthma: A Randomized, Controlled Trial Ann Intern Med. 1999;130(6):487-495. doi:10.7326/0003-4819-130-6-199903160-00005 Figure Legend: Mean percentage change from baseline (SE) in FEV 1 over the 12-week treatment period.nn± Black circles represent patients receiving montelukast, 10 mg once daily; black triangles represent patients receiving inhaled beclomethasone, 200 µg twice daily; and squares represent patients receiving placebo. The dotted lines represent the treatment effect for the subsets of patients switched from active treatment to placebo (according to the initial allocation schedule) during the washout period. White circles represent patients switched from montelukast to placebo ( = 52); white triangles represent patients switched from beclomethasone to placebo ( = 43). Date of download: Copyright The American College of Physicians. 5/10/2013 All rights reserved.
Effec2veness Trials of Montelukast Two pragma2c, open label RCTs of LTRA vs. ICS for first- line controller therapy or vs. LABA as add- on to ICS Eligible pa2ents with poor scores on Asthma Q of L or Asthma Control Ques2onnaire At 2 months and 2 years of follow up, LTRA equivalent to ICS and LABA add- on Price et al, NEJM 2011
Medicine with less efficacy in tradi2onal RCTs may have more effec2veness in Real World studies Why Might This Occur???
Less Efficacy, More Effec2veness One explana2on is greater adherence (easier to take a pill than use an inhaler) (adherence to LTRA ~65-75%; ICS ~45%) But there are others: more heterogeneous popula2on (es2mated that tradi2onal RCTs exclude 95% asthma pa2ents) role of co- medica2ons focus on pa2ent symptoms as outcomes
Efficacy vs. Effectiveness EFFICACY EFFECTIVENESS POPULATION Homogeneous Heterogeneous MEDICAL PROBLEM Narrow Disease Spectrum Broad Disease Spectrum CONDITIONS OF USE BENEFITS Strict Guidelines for Use of Main & Concomitant Medicines Hard Endpoints (Death, Major Morbidity) Real- World SoVer Endpoints (Func2oning, Symptoms, Pa2ent Preferences) Office of Technology Assessment, 1978
Key Message #1 Benefits and Harms of Medicines differ in Efficacy and Effec2veness Studies
Right Answer, Wrong Ques2on: For Observa2onal Studies Ques2on: What is the effect of postmenopausal hormone therapy on risk of coronary heart disease in postmenopausal women?
Right Answers, Wrong ques2ons: Observa2onal Studies Observa2onal studies >30% lower risk in current users compared with never users e.g., HR 0.68 in Nurses Health Study (Grodstein et al. J Women s Health 2006) Randomized trial >20% higher risk in ini2ators compared with non- ini2ators HR 1.24 in Women s Health Ini2a2ve (Manson et al. NEJM 2003)
Why Did Observa2onal Studies Disagree with RCTs? Common Explana2on: Residual Confounding (insufficient control for CHD, SES, lifestyle) (deficiencies in Observa2onal Studies) Another Explana2on: Asked Different Ques2ons
Randomized Trial: Clinically Relevant Ques2on What is CHD risk in women who ini2ate hormone treatment compared with women who ini2ate placebo or alterna2ve treatments? Design: Randomize to start hormone treatment or placebo Analysis: Compare incident (new) users and non- users
Observa2onal Studies: Clinically Irrelevant Ques2on What is risk of CHD in women currently taking hormone therapy compared with women who are not? Design: women who chose whether to use HRT Analysis: comparison of prevalent users of HRT vs. Never- Users (current vs. never users)
Current vs. Never Users Answers Irrelevant Ques2on Does not help woman wondering whether to start treatment. Does not help woman wondering whether to stop.
Women s Health Ini2a2ve vs. Nurses Health Study: New Users WHI NHS OVERALL 1.23 1.05 YRS of F/U 0-2 1.51 1.43 >2 1.07 0.91 (ITT Analysis) Hernan, et al
Ques2on MaWers More than Method RCT ques2on was clinically relevant one RCT and OS get same answer when they ask the same ques2on RESEARCH DESIGNS ARE FLAT, NOT HIERARCHICAL
KEY MESSAGE #2 BENEFITS AND HARMS DIFFER IN EFFICACY AND EFFECTIVENESS STUDIES QUESTION MATTERS MORE THAN METHOD
Right Answers, Wrong Ques2ons: For Pa2ents
What a Patient Wants " How will this medicine affect me? How will it make me feel bewer or be more ac2ve? How long will it take me to feel bewer or do more? " What are the poten2al harms and benefits if I take/switch to this medicine? " What is the best result I can expect? How will my adherence affect the results? What does high/low adherence look like? Can I be adherent to the regimen? " Is this treatment the best choice for me? Does it fit my lifestyle? Can I accept the risk/benefit balance? How will it impact my other diseases/ medicines?
Patient Centered Information: What is it? Patient Reported Outcomes (PRO) Any report of the status of a patient s health condition that comes directly from the patient, without interpretation by a clinician or anyone else. Typically a validated instrument Patient Centered Information (PCI): Clinical Dialogue Symptom Severity Daily Living Activities Physical Activity Social & Behavioral Functioning Well-Being Preferences
Prominent Organiza2ons Suppor2ng PCI IOM High Quality Health Care Patient-centered providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions Patient-Centered Outcomes Research Institute (PCORI) To help people make informed health care decisions Focusing on outcomes that people care about such as survival, function, symptoms.
Prominent Organiza2ons Suppor2ng PCI ASCO Draft Recommendations to Raise the Bar for Clinical Trials Focus on what is Clinically Meaningful Inspire Patients and Investigators to demand more from Clinical Trials Vote with you feet participate in trials that meet these recommendations
Utilization of Innovative Design & Data Collection Rigorous Qualitative Research to Identify: Patient-Centered Information Patient-Preferred Outcomes (What is most important to the patient?) Patient Centered Information Longitudinal Web-Based Data Collection Smart Phones Social media EMR (Structured Data Collection, NLP)
KEY MESSAGE #3 BENEFITS AND HARMS DIFFER IN EFFICACY AND EFFECTIVENESS STUDIES QUESTION MATTERS MORE THAN METHOD RESTORE PRIMACY OF PATIENT DIALOGUE IN DESIGN OF CLINICAL RESEARCH