A Method of Achieving Covariate Balance in Cluster Randomized Trials

Transcription

1 Insights and Innovation in Improving Health CRISP Seminar Series A Method of Achieving Covariate Balance in Cluster Randomized Trials L. Miriam Dickinson, PhD University of Colorado July 9,

2 Key Lessons Why do we randomize clusters instead of individuals? Cluster randomization: what can go wrong? Why not just use stratification? Covariate constrained randomization procedure Examples 2

3 Cluster randomized trials Unit of randomization is a group rather than an individual Groups can be defined in a variety of ways Geographic location (e.g. communities, counties, etc) Organizational units (schools/classrooms, hospitals, medical practices) Families 3

4 Why randomize clusters instead of individuals? Intervention is at the level of the group Public health educational campaign to increase colorectal cancer screening delivered to rural communities Quality improvement initiative for diabetes care delivered to primary care practices Potential contamination makes individual-level randomization problematic Feasibility convenience, economic considerations, physician preference 4

5 Common issues with CRTs The number of groups to be randomized is usually much smaller than trials in which individuals are randomized Heterogeneity among groups Individuals within groups are more similar to each other than members of other groups Simple, or even stratified, randomization of groups can result in study arms that are very different from each other in terms of individual participant characteristics More complex analyses (not addressed here) 5

6 Cluster randomization: what can go wrong? Example 1: CRT of breast cancer screening in 87 GP practices in the UK resulted in poor balance in baseline characteristics (Alexander, et al, 1989) Primary care practices were randomized to screening or control arms No data available on socio-economic levels of patients in practices prior to study initiation Screening practices had poorer socio-economic levels than controls Crude estimates of treatment effect differed considerably from adjusted estimates Authors recommended stratified approaches 6

7 Why is stratification not sufficient? Example 2: CRT to compare three approaches to implementing the chronic care model in primary care practices with the goal of improving diabetes care Randomization: Practices with similar characteristics grouped (rural/urban, practice size, and CHC vs private) and then randomized 3 waves (study arm 2 only included in 1 st two waves) No data available on diabetes process of care, limited data on patient characteristics prior to randomization 7

8 Example 2. Baseline data from a CRT using stratification Study Arm 1 15 practices, 312 patients Study Arm 2 10 practices, 189 patients Study Arm 3 15 practices, 321 patients Variable Mean or % Mean or % Mean or % Gender, % male 44.2% 52.9% 50.5% Age (years) Number of medical co-morbidities Baseline HgA1c** (n=636) Baseline systolic BP** (n=799) Baseline diastolic BP (n=799) Baseline total cholesterol* (n=703) Diabetes Process of Care* Baseline HgA1c, systolic BP, total cholesterol, and diabetes process of care all differed significantly across study arms (*p<.05, **p<.01) Implications of imbalance: Study limitation - less room for improvement in study arm 1 in primary outcome measure diabetes process of care 8

9 Covariate imbalance in randomized trials Imbalance in clinical trials is not a new problem Minimization methods for randomization of individuals were first described in the 1960 s Later extension to CRTs Raab and Butcher (2001) consider the effects of covariate imbalance on an optimal design criterion: difference between crude and adjusted treatment effect estimates Showed that differences between crude and adjusted treatment effect are minimized when differences in treatment group means on covariates to be included in the analysis are small Suggest a procedure for achieving balance Only a few CRTs have used these approaches, to date 9

10 Covariate Constrained Randomization Baseline data on units of randomization must be available Procedure 1. Generate all possible randomizations of units into study groups 2. Compute balance criterion (B), defined as the sum of squared differences between study groups on relevant standardized variables, for each randomization B=(w 1 (x 11 x 21 ) 2 + w 2 (x 12 x 22 ) 2 + ) 3. Establish a criterion for maximum allowable difference between study groups to define a set of acceptable randomizations in which the differences between treatment groups on covariates are minimized 4. Randomly select a single randomization from the set of acceptable randomizations Note: a program is available (Chaudhary), but we wrote our own to better accommodate the individual study characteristics 10

11 Study 1: Immunization in children Two reminder-recall approaches to increase up-to-date rates in month old children in Colorado Unit of randomization: county Variables for randomization: county-level data Number of children months, % up-to-date on immunizations, % in Colorado immunization registry, % black, % Hispanic, average income, ratio of pediatric to Family Medicine practices, # community health centers (CHCs) 16 counties in Colorado, stratified by rural/urban location Rural: Alamosa, Eagle, Fremont, Garfield, Grand, Logan, Otero, Rio Grand Urban: Adams, Arapahoe, Douglas, El Paso, Jefferson, Larimer, Pueblo, Weld 11

12 Study 1: Baseline Data Variable Mean Rural Min, max Mean Urban Min, max % in CIIS , , 93.0 Number of children age months % Up-to-date at baseline , , , , 54.0 % Hispanic , , 39.0 % black 1.8 0, , 10.0 Average Income ($) , Pediatric to Family Medicine ratio , , ,.61 # CHCs 2.2 0, , 11 12

13 Study 1: Methods Within rural and urban strata generate all possible randomizations using SAS Proc IML 70 possible combinations of 8 counties into 2 groups (order matters) Standardize randomization variables (z-scores) Calculate group means for each of the 70 possible randomizations For each randomization calculate balance criterion calculated (total squared difference between groups) B = (%CIISg1 - %CIISg2) 2 + (nkidsg1 nkidsg2) 2 + (UTDg1 UTDg2) 2 + (%blackg1 - %blackg2) 2 + (%HispG1 - %HispG2) 2 + (incomeg1 incomeg2) 2 + (pedsfmratiog1 pedsfmratiog2) 2 + (nchcg1 nchcg2) 2 Variables weighted equally 13

14 Study 1: Results Examined the distribution and set a balance criterion (target is approximately the best 10%) Urban: less than 1.4 as the acceptable set (slightly less than 10%) because there was a gap in the distribution at this point Rural: less than 2.0 (slightly more than 10%) Compared differences in means on raw variables for acceptable set vs others 14

15 Study 1:Distribution of balance criterion Rural Counties Urban Counties 15

16 Study 1: Magnitude + of group mean differences on raw variables: rural counties Variable Optimal randomizations Mean diff Min, max Other randomizations Mean diff Min, max % in CIIS , , 13.0 Number of children age months** % Up-to-date at baseline** 90 19, , , , 9.3 % Hispanic , , 23.3 % black , , 2.0 Average Income ($)** , , Pediatric to Family Medicine ratio.26.10, ,.40 # CHCs**.63.25, , Absolute value of differences taken for each randomization **p<.01

17 Study 1: Magnitude + of group mean differences on raw variables: urban counties Variable Optimal randomizations Mean diff Min, max Other randomizations Mean diff Min, max % in CIIS* 1.9.3, , 10.3 Number of children age months** % Up-to-date at baseline* , , , , 15.0 % Hispanic** , , 19.0 % black**.3.01, , 4.5 Average Income ($)** , , Pediatric to Family Medicine ratio.11.03, ,.24 # CHCs , , absolute value of differences taken for each randomization *p<.05, **p<.01

18 Study 1: Selected Randomization Variable Rural Group 1 Group 2 Mean Mean Urban Group 1 Group 2 Mean Mean % in CIIS Number of children age months % Up-to-date at baseline % Hispanic % black Average Income $ Pediatric to Family Medicine ratio # CHCs

19 Study 2: Immunization in ob-gyn practices To test an intervention for increasing immunization rates among eligible adult women seen in ob-gyn practices Intervention vs control practices Unit of randomization: practice Variables for randomization: practice-level data Rural/urban, # FTE clinicians, # deliveries per month, % Medicaid, current immunization practices (score for Tdap, HPV, Flu, overall score) Rural was coded 1=rural, 0 if urban Immunization scores assessed current immunization delivery for each practice 12 practices in Colorado, balanced by rural/urban location 4 rural, 8 urban 19

20 Study 2: Baseline Data Variable Mean or % Min, Max # FTE clinicians 5.8 2, 19 # deliveries ,125 % Medicaid , 42 Flu score 5.8 1, 10 HPV score 6.4 4, 9 Tdap score 2.9 0, 9 Total immunization score , 25 20

21 Study 2: Methods Generate all possible randomizations using SAS Proc IML 924 possible combinations of 12 practices into 2 groups Instead of stratifying, only randomizations with equal numbers of rural/urban practices in each group were considered 420 randomizations with 2 rural and 4 urban practices Standardize randomization variables (z-scores) Generate a file with group means for each randomization Compute balance criterion for each randomization B = (#FTEsg1 - #FTEsg2) 2 + (#deliveriesg1 #deliveriesg2) 2 + (%MedicaidG1 %MedicaidG2) 2 + (FluScoreG1 - FluScoreG2) 2 + (HPVscoreG1 - HPVscoreG2) 2 + (TdapScoreG1 TdapScoreG2) 2 + (IzScoreG1 IzScoreG2) 2 Variables weighted equally 21

22 Study 2: Results Examine the distribution of the balance criterion and set a balance criterion (target is approximately the best 10%) Balance criterion less than.85 defined as the acceptable set (10%) Compared differences in means on raw variables for acceptable set vs others 22

23 Study 2: Distribution of balance criterion 23

24 Study 2: Magnitude + of differences in group means on raw variables Variable Optimal randomizations Mean diff (SD) Min, max Other randomizations Mean diff Min, max # FTEs** 1.7.5, , 4.6 # deliveries** 8.0.2, , 42 % Medicaid** 3.5.3, , 19.7 Flu score**.7.2, , 4.5 HPV score**.3.2,.8.7.2, 1.8 Tdap score** 1.0.3, , 4.1 Overall immunization score**.9.3, , Absolute value of differences taken for each randomization **p<.01 24

25 Conclusions and Recommendations for Dissemination and Implementation Researchers Covariate constrained randomization procedures are useful to achieve balance in cluster randomized trials Preliminary data on key variables is necessary Stratification can be incorporated into the procedure Further exploration of data characteristics associated with better results is needed (e.g. underlying distributions of variables, number of units, etc) Every study is different it may not be possible to completely standardize SAS code 25

26 References Raab GM, Butcher I. Balance in cluster randomized trials. Statistics in Medicine. 2001; 20: Kraschnewski JL, Keyserling TC, Bangdiwala SI, Gizlice Z, et al. Optimized probability sampling of study sites to improve generalizability in a multisite intervention trial. Preventing Chronic Disease: Public Health Research, Practice, and Policy (1):1-8 Glynn RJ, Brookhart A, Stedman M, Avorn J, Solomon DH. Design of cluster-randomized trials of quality improvement interventions aimed at medical care providers. Medical Care (10):S2: S38-S43. Moulton LH. Covariate-based constrained randomization of grouprandomized trials. Clinical Trials 2004;1: Chaudhary MA, Moulton LH. A SAS macro for constrained randomization of group-randomized designs. Computer Methods and Programs Biomedicine : Alexander FE, Roberts MM, Lutz, W, Hepburn W. Randomization by cluster and the problem of social class bias. Journal of Epidemiology and Community Health. 1989; 43: