What is sufficient evidence to inform combination HIV prevention programs. Stefan Baral

Transcription

1 What is sufficient evidence to inform combination HIV prevention programs Stefan Baral

2 Evidence Supporting Interventions Donabedian Approach Process The Traditional Gold Standard for M&E Is system efficient? Counting the actual products distributed, people trained, etc Condoms, Peer Educators, Paralegals, etc. Structure Structural Outcomes of the Intervention Health Systems, Health Policies, etc Outcome Emerging Gold Standard What is happening with outcome of interest? Impact! Efficacy vs Effectiveness 2

3 What is Sufficient Evidence? Evidence-based medicine is a global standard Double-Blinded (DB) RCT is gold standard Evidence-based PH interventions should also be a global standard Often limited evidence, PH decision still needs to be made Precautionary Principle for PH? When there are threats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures to prevent environmental degradation To develop guidelines Need to characterize Efficacious Effective Sustainable and Scalable programs 3

4 Agency for Healthcare Research and Quality (AHRQ) US Preventive Services Task Force (USPSTF) Three-Step System Strength of Recommendation Letter (A-D, I) Level of Certainty Low, Medium, High Suggestions for Practice

5 Strength of Recommendation Rating Strength of Recommendation Practice Recommendations A Recommends the Service High Certainty that net benefit is substantial B Recommends the Service High certainty that net benefit is moderate Moderate certainty that net benefit is substantial C Recommends against Routine Provision of this service Special considerations for or against in each patient Moderate certainty that net benefit is low D Recommends against the service Moderate or high certainty that service has no net benefit or harms outweigh the benefits I Current Evidence is Insufficient to assess balance between benefits and harms Evidence is : lacking, poor quality, conflicting Offer/Provide This Service Offer/Provide This Service Offer/Provide only if special considerations support in individual Discourage the use of this service Never be offered

6 Level of Certainty Level of Certainty High Moderate Low Description Evidence from: Methodologically Sound Studies in Primary Care Populations (Generalizability) Health outcomes evaluation (effectiveness) Unlikely to be affected by future studies Enough evidence to determine effect of service on health outcomes, but limited confidence in estimate Evidence constrained by Number/size/quality of studies, Inconsistency, limited generalizability Recommendation may change based on future results Evidence is insufficient to assess health outcomes Insufficient because: Limited number/size of studies, flaws in study design, inconsistency, gaps in chain of evidence, not generalizable, inadequate info More data will allow estimation of effects on health outcomes

7 CDC Prevention Research Synthesis (PRS) Project 3 Domains Study Design Study Implementation and Analysis Strength of Evidence 2 Tiers Tier 1 Best Evidence Tier 2 Good Evidence Separate Criteria for Individual-level interventions (ILI) and group-level interventions (GLI) Community-Level Interventions (CLI)

8 CDC Prevention and Treatment of Opportunistic Infections Two Step System Letter Strength of Recommendation related to Practice Recommendation Efficacy Data Clinical Benefit Roman Numeral Quality of Evidence Source: Kaplan, et al. Guidelines for Prevention and Treatment of Opportunistic Infections in HIV-Infected Adults and Adolescents. MMWR. 2009

9 Strength of Recommendation Rating Strength of Recommendation A Strong Efficacy Data Substantial Clinical Benefit B Moderate Efficacy/Substantial Clinical Benefit High Efficacy/Limited Clinical Benefit C Insufficient Efficacy Data Good Efficacy Data/Efficacy Data does not outweigh adverse effects/actual cost/opportunity cost D Lack of Efficacy Data/Moderate adverse outcome data E Good evidence for lack of efficacy or adverse outcome Use Should always be offered Generally be offered Optional Generally not be offered Never be offered

10 Quality of Evidence Rating I II Quality of Evidence Evidence from at least one high-quality DB RCT Evidence from at least one Quasi-experimental clinical trial no randomization, no blinding, etc Cohort/Case-control data Ideally multiple centers Multiple Time-Series Studies Dramatic Results from Uncontrolled Experiments III Evidence from Expert Opinion Clinical experience Reports of Expert Committees/Authoritative Bodies Descriptive Studies

11 UK National Institute for Clinical Excellence (NICE) (Formerly the Health Development Agency)

12 Oxford Center For Evidence-Based Medicine Level of Evidence Rating within Level Details 1 A B C SR of RCTs (homogeneity) Individual RCT All or None (no outcome either before or after intervention ie parachutes) 2 A B C 3 A B SR of Cohort Studies (with homogeneity) Individual Cohort Outcomes Research/Ecological Work SR Of Case-Control Studies (with homogeneity) Individual Case-Control Studies 4 Case-Series (or poor quality cohort/case-control) 5 Expert Opinion 12

13 CEBM Grade of Recommendations Grade A B Characteristics Consistent Level 1 Study Consistent Level of 2 or 3 Studies or Extrapolating from Level 1 Studies (ie off label use) C Level 4 Studies or Extrapolations from Level 2/3 D Level 5 Evidence or troublingly inconsistent or inconclusive studies of any level 13

14 Grading of Recommendations Assessment, Development and Evaluation (GRADE) Score-based system designed for clinical interventions Type of Evidence Quality Consistency Directedness Effect Size Includes Values and Preferences PICOTS Questions Population, Intervention/Exposure, Comparison/Control, Outcome, Timing, Setting (PC, Specialty, In-Patient)

15 GRADE Characteristics Type of Evidence RCT or SR of RCT +4 Observational Evidence +2 Quality Blinding, retention, subjective outcomes 0 to -3 Directness Generalizability 0 to -2 Effect Size Measure of association >2 or >5 0 to +2

16 GRADE Score Strength of Recommendation High > or = 4 Medium 3 Low 2 Very Low < or = 1 Values and Preferences

17 Highest Attainable Standard of Evidence System for HIV Interventions (HASTE)

18 Tension Between Internal and External Validity Challenges for Evidence Combination Prevention Internal Validity Minimal study biases suggesting confidence in ultimate conclusion of the study External Validity Generalizability of ultimate findings to broader population Traditional Question for Clinicians/Programmers Does it work? What is effect size? Should I use it? Implementation Questions How, when, why, and where does it work? What factors influence effectiveness? Should I use it? How should I use it?

19 Tension in Research about Validity Traditional Approach is to establish internal validity with certain study designs and then have studies focused on external validity Internal Validity Phase 1 (Safety), Phase 2a/b (tolerability, TOC), Phase 3 (Efficacy) External Validity Phase 4 (Post-Marketing)

20 Traditional Research Pathway Effectiveness Research (and guideline development) generally happens prior to implementation research Are there more time-effective approaches to integrate implementation research with effectiveness/efficacy research Assess barriers/facilitators to intervention uptake acceptance/adoption/routinization Diagnose quality gaps Fidelity Characterize Sustainability Maintenance, Cost-Effectiveness