Introduction to Adaptive Interventions and SMAT Study Design Principles Daniel Almirall 1,2 Linda M Collins 2,3,5 Susan A Murphy 1,2,4 1 Institute for Social esearch, University of Michigan 2 The Methodology Center, Penn State University 3 Human Development and Family Studies, Pennsylvania State University 4 Department of Statistics, University of Michigan 5 Department of Statistics, Pennsylvania State University Obesity Society - Maximizing Study Design Workshop Orlando, FL - October 1, 2011 Almirall, Collins, Murphy Building Adaptive Interventions 1 / 46
Outline Adaptive Interventions What? Why? What are SMATs? Keep it Simple Choosing Primary and Secondary Hypotheses Almirall, Collins, Murphy Building Adaptive Interventions 2 / 46
I have no conflicts of interest or disclosures to make. Almirall, Collins, Murphy Building Adaptive Interventions 3 / 46
What? Why? ADAPTIVE INTEVENTIONS Almirall, Collins, Murphy Building Adaptive Interventions 4 / 46
What? Why? Definition: An Adaptive Intervention is a sequence of individually tailored decision rules that specify whether, how, and/or when to alter the intensity, type, dosage, or delivery of treatment at critical decision points in the course of care. Adaptive Interventions operationalize sequential decision making with the aim of improving clinical practice. Almirall, Collins, Murphy Building Adaptive Interventions 5 / 46
What? Why? Concrete Example of an Adaptive Intervention Pediatric Anxiety Example (SAD, GAD, SoP) First-line Txt CBT Tailoring Variable esponder Non-esponders Second-line Txt Maintain: CBT Add Treatment: Goal is to minimize the child s symptom profile/trajectory. Almirall, Collins, Murphy Building Adaptive Interventions 6 / 46
What? Why? What makes up an Adaptive Intervention? 1. Critical decisions: treatment options and more 2. Tailoring variables: to decide how to adapt treatment 3. Decision rules: inputs tailoring variable, outputs one or more recommended treatments First-line Txt CBT Tailoring Variable esponder Non-esponders Second-line Txt Maintain: CBT Add Treatment: Adaptive interventions AKA: dynamic treatment regimes, adaptive treatment strategies, treatment algorithms, structured treatment interruptions, practice parameters, ASAM criteria... Almirall, Collins, Murphy Building Adaptive Interventions 7 / 46
An Adaptive Intervention in Obesity Tailoring Variable at Intake Obesity: BMI 97%ile First line Txt: Months 0 4 GBI Tailoring Variable: ESP if 4mo Wgt Change: >10% #Family Exprmts: avg>2/q Dietary intke: avg>3fv/wk esponder Non esponder Second line Txt: Months 4 12 Maintain: GBI efreshers Augment: GBI + Personal Health Coach Obesity: BMI > 97%ile GBI + Personal Health Coach esponder Non esponder Step Down: PHC lite (phone) Augment: GBI + Personal Health Coach + Medication
What? Why? Why Adaptive Interventions? Necessary because... Chronic nature of substance use/mental health disorders Waxing and waning course (multiple relapse, recurrence) Genetic and non-genetic factors influence course Co-occuring disorders may arise High patient heterogeneity in response to treatment Within person (over time) differential response to treatment Between person differential response to treatment All require sequences of treatment decisions. Almirall, Collins, Murphy Building Adaptive Interventions 9 / 46
GENEATING HYPOTHESES vs BUILDING vs EVALUATING ADAPTIVE INTEVENTIONS? Almirall, Collins, Murphy Building Adaptive Interventions 10 / 46
3 Different esearch Questions/Aims = 3 Different esearch Designs Aim 1: When generating hypotheses to build an Adaptive Intervention: e.g., Does augmenting txt (as observed in a previous trial) for non-responders correlate with better outcomes? Aim 2: When building an Adaptive Intervention: e.g, What are the best tailoring variables and/or decision rules? Aim 3: When evaluating a particular Adaptive Intervention: e.g. Does the AI have a statistically significant effect as compared to control intervention? Almirall, Collins, Murphy Building Adaptive Interventions 11 / 46
3 Different esearch Questions/Aims = 3 Different esearch Designs Ex. Q1: Does augmenting txt for non-responders (as observed in a previous trial) correlate with better outcomes? Ex. Q2: What are the best tailoring variables or decision rules? Ex. Q3: Does the Adaptive Intervention have a statistically significant effect as compared to control intervention? Observational Experimental Studies Studies e.g., Analysis of Question Aim Previous CT SMAT CT 1 Hypothesis Gen. YES no 2 Building YES 3 Evaluating no YES
What are SMATs? SEQUENTIAL MULTIPLE ASSIGNMENT ANDOMIZED TIALS (SMATs) Almirall, Collins, Murphy Building Adaptive Interventions 13 / 46
What are SMATs? What is a Sequential Multiple Assignment andomized Trial (SMAT)? Multi-stage trials; same participants throughout Each stage corresponds to a critical decision point At each stage, subjects randomized to set of treatment options The goal of a SMAT is to inform the development of adaptive interventions. I will give you an example SMAT, but first... Almirall, Collins, Murphy Building Adaptive Interventions 14 / 46
What are SMATs? Motivation for an Example SMAT Child-Adolescent Anxiety Multi-modal Study (CAMS) CAMS: acute-phase, efficacy, CT for child anxiety CBT+MED > MED CBT > Placebo However, some families and clinicians remain concerned about the use of MED in this population So an important next question for clinical practice is Can we delay the use of MED? If so, for whom? Some children may do fine w/ CBT only and not need MED. Almirall, Collins, Murphy Building Adaptive Interventions 15 / 46
Concrete Example of a SMAT: Pediatric Anxiety Courtesy of Scott N Compton, Duke University Medical Center O1 O2 + Primary First-line Txt Second-line Txt Y Tailoring Variable CBT Non-esponders esponders esponders Non-esponders Add Treatment: + FT Maintain: Step Down: CBT Only Maintain: CBT Add Treatment: Switch Treatment: MED
One Adaptive Intervention Within the SMAT O1 O2 + Primary First-line Txt Second-line Txt Y Tailoring Variable CBT Non-esponders esponders esponders Non-esponders Add Treatment: + FT Maintain: Step Down: CBT Only Maintain: CBT Add Treatment: Switch Treatment: MED
Another Adaptive Intervention Within the SMAT O1 O2 + Primary First-line Txt Second-line Txt Y Tailoring Variable CBT Non-esponders esponders esponder Non-esponders Add Treatment: + FT Maintain: Step Down: CBT Only Maintain: CBT Add Treatment: Switch Treatment: MED
4 Embedded Adaptive Interventions in this SMAT AI 1 Non esponders esponders Add Treatment: + FT Step Down: CBT Boost AI 2 Non esponders esponders Add Treatment: + FT Maintain: AI 3 CBT esponders Non esponders Maintain: CBT Boost Add Treatment: AI 4 CBT esponders Non esponders Maintain: CBT Boost Switch Treatment: MED
Keep it Simple Choosing Primary and Secondary Hypotheses SMAT DESIGN PINCIPLES Almirall, Collins, Murphy Building Adaptive Interventions 20 / 46
Keep it Simple Choosing Primary and Secondary Hypotheses KISS Principle: Keep It Simple, Straightforward Power for simple important primary hypotheses Take Appropriate steps to develop an more deeply-individualized Adaptive Intervention Almirall, Collins, Murphy Building Adaptive Interventions 21 / 46
Keep it Simple Choosing Primary and Secondary Hypotheses Keep It Simple, Straightforward Overarching Principle At each stage, or critical decision point,... Use low dimensional summary to restrict subsequent treatments Use binary responder status Should be easy to use in actual clinical practice estrict class of treatment options only by ethical, feasibility, or strong scientific considerations Collect additional, auxiliary time-varying measures To develop a more deeply-tailored Adaptive Intervention Think time-varying effect moderators Almirall, Collins, Murphy Building Adaptive Interventions 22 / 46
Keep it Simple Choosing Primary and Secondary Hypotheses SMAT Design: Primary Aims Choose a simple primary aim/question that aids development of an adaptive intervention. Power the SMAT to test this hypothesis. Almirall, Collins, Murphy Building Adaptive Interventions 23 / 46
Primary Aim Example 1 What is the main effect of first-line treatment? End of study outcome (e.g., ANOVA). O1 Power ES N 0.8 52 0.5 128 0.2 788 α = 0.05 β = 0.20 O2 + Primary First-line Txt Second-line Txt Y Tailoring Variable CBT Non-esponders esponders esponders Non-esponders Add Treatment: + FT Maintain: Step Down: CBT Only Maintain: CBT Add Treatment: Switch Treatmnt: MED
Primary Aim Example 2 What is the main effect of first-line treatment? Longitudinal outcome (e.g., LMM). O1 Power ES N 0.8 34 0.5 83 0.2 505 ρ = 0.60 α = 0.05 β = 0.20 O2 + Primary First-line Txt Second-line Txt Y Tailoring Variable CBT Non-esponders esponders esponders Non-esponders Add Treatment: + FT Maintain: Step Down: CBT Only Maintain: CBT Add Treatment: Switch Treatmnt: MED
Keep it Simple Choosing Primary and Secondary Hypotheses Do not use Early Non/esponse as Primary Outcome Why choose a longitudinal or end-of-study outcome, or a with-in person summary of outcomes over time? These are chronic disorders Outcome should incorporate time to initial response as a component Quick initial relief of symptoms should be valued Examples: growth (slope), pre-post change, survival, end of study, AUC Almirall, Collins, Murphy Building Adaptive Interventions 26 / 46
Keep it Simple Choosing Primary and Secondary Hypotheses SMAT Design: Secondary Aims Choose secondary aims/questions that further develop the Adaptive Intervention and take advantage of sequential randomization to eliminate confounding. Almirall, Collins, Murphy Building Adaptive Interventions 27 / 46
Keep it Simple Choosing Primary and Secondary Hypotheses Secondary Aim Examples 1 and 2 Best second-line treatment and second-line treatment tailoring aim. O2 + Primary First-line Txt Second-line Txt Y Tailoring Variable CBT Non-esponders O2 = CBT adherence, time to non-response, allegiance with therapist, changes in home environment Add Treatment: Switch Treatment: MED Almirall, Collins, Murphy Building Adaptive Interventions 28 / 46
Secondary Aim Example 3 Build a more deeply tailored adaptive intervention. O1 CBT O1 = demographics, genetics, subdiagnoses, comorbidities, etc O2 + Primary First-line Txt Second-line Txt Y Tailoring Variable Non-esponders esponders esponders Non-esponders O2 = adherence, time to N, changes at home, etc Add Treatment: + FT Maintain: Step Down: CBT Only Maintain: CBT Add Treatment: Switch Treatment: MED
DISCUSSION Almirall, Collins, Murphy Building Adaptive Interventions 30 / 46
Adaptive Interventions vs Adaptive Designs? These ideas are not directly related. Confusing! Problem 1: word design often used in 2 different ways Intervention design Experimental design Problem 2: word adaptive often used in 2 different ways Is experimental design adaptive? Is experimental design used to build an adaptive intervention? Almirall, Collins, Murphy Building Adaptive Interventions 31 / 46
Adaptive Interventions vs Adaptive Designs? Adaptive interventions are a type of intervention design Adaptive experimental designs are particular type of experimental design SMATs are not Adaptive Experimental Designs SMATs do inform development of Adaptive Interventions Almirall, Collins, Murphy Building Adaptive Interventions 32 / 46
Take Home the Following Adaptive Interventions individualize treatment up-front and throughout Adaptive Interventions are guides for clinical practice SMATs are used to build better Adaptive Interventions Next trial: SMAT-optimized Adaptive Intervention vs. state-of-the-art treatment SMATs do not necessarily require larger sample sizes Existing CTs can be used to begin to learn about adaptive interventions Observational study Almirall, Collins, Murphy Building Adaptive Interventions 33 / 46
Adaptive Treatment for Children with ADHD PI: Pelham, Florida International University esponders Continue Medication Medication Behavioral Intervention Non-esponders esponders Non-esponders Increase Medication Dose Add Behavioral Intervention Continue Behavioral Intervention Increase Behavioral Intervention Add Medication
Treatment for Alcohol Dependence PI: Oslin, University of Pennsylvania Early Trigger for N: 2+ HDD CBI Non-esponse Late Trigger for N: 5+ HDD 8 Week esponse 8 Week esponse Non-esponse Naltrexone TDM + Naltrexone CBI + Naltrexone Naltrexone TDM + Naltrexone CBI CBI + Naltrexone
Thank you! Questions? Find papers on SMAT: http://www.lsa.stat.umich.edu/ samurphy/ (Susan Murphy) http://methcenter.psu.edu (Linda Collins) These slides will be posted on my website: http://www-personal.umich.edu/ dalmiral/ Email me with questions about this presentation: Daniel Almirall: dalmiral@umich.edu Almirall, Collins, Murphy Building Adaptive Interventions 36 / 46
Extra Slides Almirall, Collins, Murphy Building Adaptive Interventions 37 / 46
Hypothesis-generating Observational Studies Post-hoc Analyses Useful for Building Adaptive Interventions Give examples of different observational study questions they can examine using data from a previous 2-arm CT Standard observational study caveats apply: No manipulation usually means lack of heterogeneity in txt options (beyond what is controlled by experimentation in original CT) Some CTs use samples that are too homogeneous Confounding by observed baseline and time-varying factors Unobserved, unknown, unmeasured confounding by baseline and time-varying factors Almirall, Collins, Murphy Building Adaptive Interventions 38 / 46
Hypothesis-generating Observational Studies Post-hoc Analyses Useful for Building Adaptive Interventions There exists a literature for examining the impact of time-varying treatments in observational studies Marginal Structural Models (obins, 1999; Bray, Almirall, et al., 2006) to examine the marginal impact of observed time-varying sequences of treatment Structural Nested Mean Models (obins, 1994; Almirall, et al., 2010, 2011) to examine time-varying moderators of observed time-varying sequences of treatment Marginal Mean Models (Murphy, et al., 2001): to examine the impact of observed adaptive interventions Almirall, Collins, Murphy Building Adaptive Interventions 39 / 46
Early precursors to SMAT CATIE (2001) Treatment of Psychosis in Patients with Alzheimer s CATIE (2001) Treatment of Psychosis in Patients with Schizophrenia STA*D (2003) Treatment of Depression Almirall, Collins, Murphy Building Adaptive Interventions 40 / 46
Other Alternatives Piecing Together esults from Multiple Trials Choose best first-line treatment on the basis of a two-arm CT; then choose best second-line treatment on the basis of another separate, two-arm CT Concerns: delayed therapeutic effects, and cohort effects Observational (Non-experimental) Comparisons of AIs Using data from longitudinal randomized trials May yield results that inform a SMAT proposal Understand current treatment sequencing practices Typical problems associated with observational studies Expert Opinion Almirall, Collins, Murphy Building Adaptive Interventions 41 / 46
Why Not Use Multiple Trials to Construct an AI Three Concerns about Using Multiple Trials as an Alternative to a SMAT 1. Concern 1: Delayed Therapeutic Effect 2. Concern 2: Diagnostic Effects 3. Concern 3: Cohort Effects All three concerns emanate from the basic idea that constructing an adaptive intervention based on a myopic, local, study-to-study point of view may not be optimal. Almirall, Collins, Murphy Building Adaptive Interventions 42 / 46
Why Not Use Multiple Trials to Construct an AI Concern 1: Delayed Therapeutic Effects, or Sequential Treatment Interactions Positive Synergy Btwn First- and Second-line Treatments Tapering off medication after 12 weeks of use may not appear best initially, but may have enhanced long term effectiveness when followed by a particular augmentation, switch, or maintenance strategy. Tapering off medication after 12 weeks may set the child up for better success with any one of the second-line treatments. Almirall, Collins, Murphy Building Adaptive Interventions 43 / 46
Why Not Use Multiple Trials to Construct an AI Concern 1: Delayed Therapeutic Effects, or Sequential Treatment Interactions Negative Synergy Btwn First- and Second-line Treatments Keeping the child on medication an additional 12 weeks may produce a higher proportion of responders at first, but may also result in side effects that reduce the variety of subsequent treatments available if s/he relapses. The burden associated with continuing medication an additional 12 weeks may be so high that non-responders will not adhere to second-line treatments. Almirall, Collins, Murphy Building Adaptive Interventions 44 / 46
Why Not Use Multiple Trials to Construct an AI Concern 2: Diagnostic Effects Tapering off medication after 12 weeks initial use may not produce a higher proportion of responders at first, but may elicit symptoms that allow you to better match subsequent treatment to the child. The improved matching (personalizing) on subsequent treatments may result in a better response overall as compared to any sequence of treatments that offered an additional 12 weeks of medication after the initial 12 weeks. Almirall, Collins, Murphy Building Adaptive Interventions 45 / 46
Why Not Use Multiple Trials to Construct an AI Concern 3: Cohort Effects Children enrolled in the initial and secondary trials may be different. Children who remain in the trial(s) may be different. Characteristics of adherent children may differ from study to study. Children that know they are undergoing adaptive interventions may have different adherence patterns. Bottom line: The population of children we are making inferences about may simply be different from study-to-study. Almirall, Collins, Murphy Building Adaptive Interventions 46 / 46