Lisa Yelland. BMa&CompSc (Hons)

Transcription

1 Statistical Issues Associated with the Analysis of Binary Outcomes in Randomised Controlled Trials when the Effect Measure of Interest is the Relative Risk Lisa Yelland BMa&CompSc (Hons) Discipline of Public Health School of Population Health and Clinical Practice Faculty of Health Sciences The University of Adelaide Australia Thesis submitted in fulfillment of the requirements for the degree of Doctor of Philosophy, September 2011

2 Contents Abstract... vi Declaration... viii Manuscripts Contributing to this Thesis... x Presentations Arising out of this Thesis... xi Awards Arising out of this Thesis... xii Acknowledgements... xiii Abbreviations...xv 1 Introduction Problems with Odds Ratios Advantages of Relative Risks Problems with Relative Risks Thesis Aim Thesis Outline Literature Review and Aims Independent Data Methods for Estimating Relative Risks Comparison of Methods Specific Aim Clustered Data Analysing Clustered Data Simulating Clustered Binary Outcomes Specific Aim i

3 2.2.4 Methods for Estimating Relative Risks Specific Aim Perinatal Trials Multiple Births Specific Aim Methods Summarising Simulation Results Common Approaches Approaches Used in this Thesis Computational Resources Simulating Clustered Data Avoiding Invalid Success Probabilities Relative Risk Estimation in Randomised Controlled Trials: A Comparison of Methods for Independent Observations Preface Statement of Authorship Article Abstract Introduction Methods Simulation Results Example Discussion Recommendations Additional Discussion...69 ii

4 5 Adjusted Intraclass Correlation Coefficients for Binary Data: Methods and Estimates from a Cluster Randomised Trial in Primary Care Preface Statement of Authorship Article Abstract Introduction Methods Results Discussion Additional Discussion Relative Risk Estimation in Cluster Randomised Trials: A Comparison of Generalised Estimating Equation Methods Preface Statement of Authorship Article Abstract Introduction Methods Simulation Results Example Discussion Additional Discussion Performance of the Modified Poisson Regression Approach for Estimating Relative Risks with Clustered Prospective Data Preface iii

5 7.2 Statement of Authorship Article Abstract Introduction Methods Results Illustrative Examples Discussion Additional Discussion Analysis of Binary Outcomes from Randomised Trials Including Multiple Births: When Should Clustering be Taken Into Account? Preface Statement of Authorship Article Abstract Introduction Methods Results Discussion Additional Discussion Relative Risks Summary and Conclusions Key Findings and Contributions Specific Aim Specific Aim iv

6 9.1.3 Specific Aim Specific Aim Limitations and Future Directions Concluding Remarks References v

7 Abstract Background: Binary outcomes have traditionally been analysed using logistic regression which estimates odds ratios. A popular alternative is to estimate relative risks using log binomial regression. Due to convergence problems with this model, alternative methods have been proposed for estimating relative risks. Comparisons between methods are limited and guidance on which method(s) should be used in practice is lacking. These methods are often applied to clustered data, despite the absence of evidence supporting their use in this setting. Comparison of methods in the clustered data setting via simulation is difficult. The simulation model requires specification of the random effects variance on the log scale, but the intraclass correlation coefficient (ICC) on the probability scale is the preferred measure of dependence. The relationship between the ICC and the random effects variance has been defined under the logistic model but not the log binomial model. The appropriate method for analysing binary outcomes from perinatal trials which include infants from multiple births is a matter of debate, and relative risks have received little attention in this context. Aim: To investigate statistical issues associated with the analysis of binary outcomes in randomised controlled trials (RCTs) when the effect measure of interest is the relative risk. Specifically, the aims are: To compare the performance of methods for estimating relative risks in RCTs with independent and clustered observations; To determine the relationship between the ICC on the probability scale and the between cluster variance on the log scale; To provide guidance on the analysis of binary outcomes from perinatal trials including infants from multiple births. vi

8 Methods: Simulation studies are conducted to compare methods for estimating relative risks using independent and clustered data. To determine the ICC in the latter scenario, the relationship between the ICC on the probability scale and the random effects variance on the log scale is derived. Additional simulation studies are conducted to determine how different analytical methods compare in perinatal trials with multiple births. Example datasets are analysed for illustration. Results: Some methods for estimating relative risks are associated with large bias and poor coverage. Others fail to overcome the convergence problems of log binomial regression. Several methods perform well across a wide range of independent and clustered data settings, including modified Poisson regression. When simulating clustered data, the ICC can be determined from the random effects variance on the log scale based on a Taylor series expansion or properties of the lognormal distribution. Failure to account for clustering in perinatal trials including multiple births leads to inflated type I errors and undercoverage, unless both the ICC and the multiple birth rate are low. Conclusion: Relative risks are a useful measure of effect for binary outcomes. Difficulties in estimating relative risks due to convergence problems with log binomial regression can be overcome using one of several alternatives, including the popular modified Poisson regression approach. This method works well for both independent and clustered data. Clustering should be taken into account in the analysis of perinatal trials including multiple births. vii

9 Declaration This work contains no material which has been accepted for the award of any other degree or diploma in any university or other tertiary institution to Lisa Yelland and, to the best of my knowledge and belief, contains no material previously published or written by another person, except where due reference has been made in the text. I give consent to this copy of my thesis when deposited in the University Library, being made available for loan and photocopying, subject to the provisions of the Copyright Act The author acknowledges that copyright of published works contained within this thesis (as listed below) resides with the copyright holder(s) of those works. Yelland LN, Salter AB, Ryan P. Relative risk estimation in randomized controlled trials: a comparison of methods for independent observations. The International Journal of Biostatistics 2011; 7(1):5. DOI: / Berkeley Electronic Press. Yelland LN, Salter AB, Ryan P, Laurence CO. Adjusted intraclass correlation coefficients for binary data: methods and estimates from a cluster randomized trial in primary care. Clinical Trials 2011; 8: DOI: / The Author(s), Yelland LN, Salter AB, Ryan P. Relative risk estimation in cluster randomized trials: a comparison of generalized estimating equation methods. The International Journal of Biostatistics 2011; 7(1):27. DOI: / Berkeley Electronic Press. Yelland LN, Salter AB, Ryan P. Performance of the modified Poisson regression approach for estimating relative risks with clustered prospective data. American Journal of Epidemiology: first published online August 12, DOI: /aje/kwr183. The Author viii

10 Yelland LN, Salter AB, Ryan P, Makrides M. Analysis of binary outcomes from randomised trials including multiple births: when should clustering be taken into account? Paediatric and Perinatal Epidemiology 2011; 25: DOI: /j x Blackwell Publishing Ltd. I also give permission for the digital version of my thesis to be made available on the web, via the University s digital research repository, the Library catalogue, the Australasian Digital Theses Program (ADTP) and also through web search engines, unless permission has been granted by the University to restrict access for a period of time. Signed: Date: Lisa Yelland (Candidate) ix

11 Manuscripts Contributing to this Thesis Yelland LN, Salter AB, Ryan P. Relative risk estimation in randomized controlled trials: a comparison of methods for independent observations. The International Journal of Biostatistics 2011; 7(1):5. DOI: / Yelland LN, Salter AB, Ryan P, Laurence CO. Adjusted intraclass correlation coefficients for binary data: methods and estimates from a cluster randomized trial in primary care. Clinical Trials 2011; 8: DOI: / Yelland LN, Salter AB, Ryan P. Relative risk estimation in cluster randomized trials: a comparison of generalized estimating equation methods. The International Journal of Biostatistics 2011; 7(1):27. DOI: / Yelland LN, Salter AB, Ryan P. Performance of the modified Poisson regression approach for estimating relative risks with clustered prospective data. American Journal of Epidemiology: first published online August 12, DOI: /aje/kwr183 Yelland LN, Salter AB, Ryan P, Makrides M. Analysis of binary outcomes from randomised trials including multiple births: when should clustering be taken into account? Paediatric and Perinatal Epidemiology 2011; 25: DOI: /j x x

12 Presentations Arising out of this Thesis Yelland LN. Analysis of Perinatal Trials Including Multiple Births: When Should Clustering be Taken Into Account? Statistical Society of Australia and the Australasian Epidemiological Association Joint Seminar Series. Adelaide, July Yelland LN. Estimating Relative Risks from Clustered Data. School of Population Health and Clinical Practice Seminar Series. Adelaide, December Yelland LN. Clustering in Perinatal Trials: What is it and when does it matter? School of Population Health and Clinical Practice HDR Research Symposium. Adelaide, Yelland LN, Salter AB, Ryan P. Relative Risk Estimation in Randomised Controlled Trials: A Comparison of Methods for Independent Observations. International Biometrics Society Australasian Region Conference. Taupo, New Zealand, December Yelland LN. Relative Risk Estimation in Randomised Controlled Trials: A Comparison of Methods for Independent Observations. School of Population Health and Clinical Practice Seminar Series. Adelaide, July Yelland LN. Statistical Issues Associated with Binary Outcome Data in Randomised Controlled Trials. School of Population Health and Clinical Practice Seminar Series. Adelaide, August xi

13 Awards Arising out of this Thesis Overall best presentation, School of Population Health and Clinical Practice HDR Research Symposium, Adelaide, Runner up, best presentation by a young statistician, International Biometric Society Australasian Region conference in Taupo, New Zealand, CSIRO scholarship to attend the International Biometric Society Australasian Region conference in Taupo, New Zealand, xii

14 Acknowledgements I would like to sincerely thank the following people for helping to make this thesis possible. To my supervisors, Phil Ryan and Amy Salter, thank you for all your advice, support and encouragement throughout my candidature. You have both taught me many things in the years we have worked together that I know will stay with me throughout my career. Special thanks must also go to Phil for possibly being the fastest PhD supervisor on the planet at providing feedback! To Maria Makrides, thank you for your involvement with the multiple births paper, and for giving me the opportunity to be involved with some fantastic projects outside this thesis. My interest in and passion for perinatal trials has been born out of these experiences and I am grateful for your role in that. To Caroline Laurence, thank you for your involvement with the ICC paper and for all you taught me during our years working on the PoCT Trial, particularly about writing papers. Those lessons have been helpful throughout this thesis and no doubt will continue to be of great use in future. To Justin Beilby on behalf of the PoCT Trial Management Committee, Maria Makrides on behalf of the DINO Trial Steering Committee and Carmel Collins, thank you for granting me permission to use your datasets as examples in this thesis. To my fellow PhD candidates, thank you for sharing this exciting and challenging journey with me. Special thanks to Nicole for going first and sharing your wisdom with me, to Jesia and Oana for always being available for a chat, and to my office mates George and Marianne for your support and for creating a quiet and productive work environment (most of the time!). To my friends and family, thank you for your support and the much needed distractions throughout this journey. xiii

15 To my parents, thank you for encouraging me to do my best, always believing in me, and giving me all the best opportunities in life. I would not be where I am today without your love and support. Finally to Tom, thank you for sharing every high and low of this rollercoaster ride I have been on for the last three years. You have talked through the issues with me, listened to my frustrations, reminded me there s more to life than this thesis and celebrated all the wins with me. I am so lucky to have shared this journey with you. xiv

16 Abbreviations ACR AGHQ ANOVA CS DEFF DHA GEE GLM HbA1c HDL ICC INR IQR MEM MQL MS PoCT PQL RCT Albumin-Creatinine Ratio Adaptive Gauss-Hermite Quadrature Analysis of Variance Conditional Standardisation Design Effect Docosahexaenoic Acid Generalised Estimating Equation Generalised Linear Model Glycated Haemoglobin High Density Lipoprotein Intraclass/Intracluster Correlation Coefficient International Normalised Ratio Interquartile Range Mixed Effects Model Marginal Quasi-Likelihood Marginal Standardisation Point of Care Testing Penalised Quasi-Likelihood Randomised Controlled Trial xv

17 1 Introduction Binary outcomes are routinely encountered in randomised controlled trials (RCTs). For example, when assessing safety it is common to record whether a subject experiences a particular serious adverse event, such as hospitalisation or death. Many efficacy outcomes are also binary in nature, for instance, the primary outcome for the Point of Care Testing (PoCT) Trial [1]. In this trial, pathology test results were determined for patients using either point of care testing machines or pathology laboratories, and the primary outcome was whether the test result was within or outside a pre-specified target range. Such outcomes may be represented by Y i for subject i ( i 1,, n), where Y 1 if the subject experiences the outcome of interest (e.g. test result is within the target range), and Yi 0 otherwise. i Binary outcomes have traditionally been analysed using the logistic regression model where E Y i i log 1 X X X, (1.1) i i i i K Ki is the probability of experiencing the outcome of interest for subject i, and X, X, X are predictor variables. This model is a member of the class of generalised linear 1i 2i Ki models (GLMs) [2] and combines a logit link with a binomial distribution. In RCTs, X 1i is often a treatment group indicator, coded as X1 i 1 for subjects in the treatment group and X1 i 0 for subjects in the control group, while X 2, X are typically baseline covariates and/or i stratification variables. The treatment effect is described in terms of an odds ratio, given by exp 1 Ki, which is interpreted as the odds of experiencing the outcome of interest for an intervention subject relative to a control subject. 1.1 Problems with Odds Ratios One disadvantage of the odds ratio as a measure of effect is that it is considered difficult to interpret [3-5]. As a result, it is often misinterpreted as a relative risk [4, 6]. A famous example of this comes from a study examining the effect of gender and race on physicians 1

18 recommendations for patients exhibiting chest pain [7]. Physicians were presented with medical information as well as a recorded interview for hypothetical patients, and were asked to make recommendations for treating each patient. The outcome of interest was whether the physician referred the patient for cardiac catheterisation. Results were presented as odds ratios comparing female patients to male patients, and black patients to white patients. The odds ratio was 0.6 in each case. The results of this study received a great deal of attention in the media, where odds ratios were incorrectly interpreted as relative risks. This led to reports that females and black patients were 40% less likely to be referred for cardiac catheterisation compared to males and white patients respectively. In fact, the relative risk was 0.93 in each case, indicating that females and black patients were only 7% less likely to be referred for cardiac catheterisation [8]. By interpreting these odds ratios as relative risks, the effects of both gender and race on physicians recommendations were severely overstated in the media. Another disadvantage of the odds ratio is that it is not collapsible [9-10]. In other words, the odds ratio can be identical in each stratum defined by the levels of a covariate which is not a confounder in the usual sense, but the overall odds ratio can differ from the stratum-specific odds ratios. As an example of this phenomenon, consider the results of a hypothetical RCT to determine the effect of a new study technique believed to improve the chance of achieving an A grade on exams (Table 1.1). A total of 400 students are recruited, with half assigned to the new study technique (intervention) and the remainder assigned to the usual study technique (control). Part-time job status is collected, since having a part-time job is believed to be related to the outcome. This is evident in the trial, as only 30% (30/100) of students with a part-time job receive an A grade compared to 70% (210/300) of students without a part-time job. Since parttime job status is balanced between treatment groups, with 25% (50/200) of students in each group having a part-time job, it does not meet the usual criteria for confounding [11]. When considering students with and without a part-time job separately, the odds ratio comparing intervention to control is 2.67 in each case. Ignoring part-time job status results in an overall odds ratio of 2.33, which is somewhat lower than the stratum-specific odds ratios. 2

19 Table 1.1: Results from a hypothetical RCT to determine the effect of a new study technique compared to the usual study technique on exam results, both overall and stratified by part-time job status. A Grade Not A Grade Total Odds Ratio Part-time job Intervention Control Total No part-time job Intervention Control Total Combined Intervention Control Total More extreme examples of non-collapsibility can be found in the literature. For instance, Miettinen and Cook [9] present a hypothetical example where the outcome (illness) depends on an exposure. The gender-specific odds ratios comparing exposed to unexposed patients are 5.2, but the overall odds ratio ignoring gender is only 1.2. In both this and the previous example, the overall odds ratio is closer to the null value of 1 compared to the stratum-specific values. In a RCT, where covariates are expected to be balanced between treatment groups due to randomisation, omitting a covariate from the logistic regression model that is related to the outcome results in an estimated treatment effect that is biased towards the null [12-14]. The magnitude of the bias increases as either the effect of the covariate on the outcome increases, or the variance of the covariate increases [13]. This highlights the importance of identifying and controlling for covariates that are related to the outcome when the odds ratio is chosen as the effect measure of interest. Controlling for such covariates can present difficulties in practice, however, due to restrictions on the number of covariates that can feasibly be accounted for in the model, as well as limitations in the current state of knowledge about covariates that influence the outcome of interest. A related difficulty with odds ratios occurs in the presence of clustered data. Clustering violates the usual assumption of independent observations required for ordinary logistic 3

20 regression to be valid. In order to account for clustering, both mixed effects models (MEMs) and generalised estimating equations (GEEs) are commonly used. MEMs account for clustering through the inclusion of a random cluster effect in the analysis model [15], while GEEs allow observations within a cluster to be correlated by specifying a working correlation structure [16]. The random cluster effect in the MEM can be thought of as an omitted covariate from the GEE, indicating that odds ratio from GEEs will be closer to the null than odds ratios from MEMs [17]. Care must be taken when interpreting odds ratio estimates based on clustered data and when making comparisons between estimates from different studies, since odds ratios obtained from GEEs and MEMs require different interpretations and are not directly comparable [17-18]. This issue is covered in more detail in Section Advantages of Relative Risks Given the potential difficulties with the odds ratio discussed above, alternative measures of effect for binary outcomes are often of interest. A popular alternative is the relative risk. In contrast to the odds ratio, the relative risk is considered simple to interpret [3-5, 19]. In a RCT, the relative risk describes the risk of experiencing the outcome of interest for an intervention subject relative to a control subject. The relative risk is also collapsible [9-10] and no bias is introduced when an important covariate is omitted from the analysis model, provided it is unrelated to treatment [13-14]. This means that unadjusted and adjusted estimates of relative risk in RCTs both estimate the same treatment parameter, which simplifies interpretation and makes covariate selection for adjusted analyses less important, provided the treatment groups are well balanced with respect to covariates. A further advantage of the relative risk is that GEEs and MEMs estimate the same relative risk for treatment in the presence of clustered data [17-18], thus reducing the potential for misinterpreting the results or making inappropriate comparisons between estimates from different studies. Given these advantages of relative risks over odds ratios, many authors have expressed a preference for the relative risk as a measure of effect for binary outcomes [3-4, 6, 10]. 4

21 1.3 Problems with Relative Risks Despite the many advantages of the relative risk over the odds ratio, this measure of effect is not without its limitations. The main disadvantage of the relative risk is that it can be difficult to estimate when adjustment for covariates is of interest. Although adjustment is not required to obtain an unbiased estimate of treatment effect in RCTs provided the treatment groups are well balanced with respect to covariates [14], it can lead to a more efficient estimate [20-21]. Adjusted relative risks can be estimated easily from the logistic regression model if the outcome of interest is rare, since the odds ratio approximates the relative risk in this case [22]. However, if the outcome of interest is common, the odds ratio no longer approximates the relative risk and an alternative method of estimation is required. To estimate relative risks directly, the log binomial regression model may be used: where i and X1, X 2, X i i Ki log i X i X i K X Ki, (1.2) are as defined previously. Under this model, the treatment relative risk is given by exp 1. Like the logistic regression model, the log binomial regression model is a member of the class of GLMs but it assumes a log link, rather than a logit link. Since i is a probability which must lie between zero and one, the left-hand side of model (1.2) is constrained to be less than or equal to zero, while the right-hand side is unconstrained. This inconsistency can lead to convergence problems with the model. No such problems occur for the logistic regression model, since both sides of model (1.1) are unconstrained. Due to the potential for convergence problems with log binomial regression, other methods for estimating relative risks must be considered. Many alternative methods have been suggested [3, 22-34], as will be described in Section 2.1, but comparisons between methods have been limited. In addition, the performance of these methods for analysing clustered data has received limited attention, despite the fact that they are often used to estimate relative risks in this setting. 5

22 1.4 Thesis Aim The general aim of this thesis is to investigate statistical issues associated with the analysis of binary outcomes in RCTs when the effect measure of interest is the relative risk. As this aim is very broad, four specific aims were identified: To compare the performance of methods for estimating relative risks in the context of RCTs with independent observations, and to make recommendations about which method(s) should be used in practice. To determine the relationship between the ICC on the probability scale and the between cluster variance on the log scale for clustered binary outcomes. To compare the performance of methods for estimating relative risks in the context of RCTs with clustered observations, and to make recommendations about which method(s) should be used in practice. To determine if and when clustering should be taken into account in the analysis of binary outcomes from perinatal trials including infants from multiple births, and whether the choice of effect measure influences the results. Full details, including the motivation for each of these specific aims, are given in Chapter Thesis Outline The remainder of this thesis is organised as follows. In Chapter 2, I review the relevant literature to motivate each of the four specific aims of this thesis, introduced above. Different methods for estimating relative risks are discussed generally in the context of independent data, and issues associated with applying these methods to clustered data are considered. The focus then moves to perinatal trials as an area of application. Controversy exists regarding the best approach for analysing binary outcomes from perinatal trials when infants from multiple births (e.g. twins) are enrolled in the trial. 6

23 To address each of the specific aims of this thesis, I conducted several large simulation studies. In Chapter 3, I discuss general methodological issues related to conducting these simulation studies. These issues include choosing an approach for summarising the large volumes of data produced by the simulation studies in a concise and informative way, and making the most of available computational resources. Details of the actual simulation studies conducted as part of this thesis are included in subsequent chapters. Publications arising from this thesis, each of which addresses one of the four specific aims, are included in Chapters 4 to 8. Each chapter relates to the analysis of binary outcomes in RCTs when the effect measure of interest is the relative risk but the context varies. In Chapter 4, the focus is on independent observations. Clustered observations are discussed in Chapters 5 to 7, while perinatal trials are considered in Chapter 8. Finally, a general discussion of the results, suggestions for future research, and concluding remarks are given in Chapter 9. 7

24 2 Literature Review and Aims In this chapter, I review the literature to identify current gaps in knowledge relating to the analysis of binary outcome in RCTs, and to motivate each of the four specific aims that will be addressed in this thesis. The chapter begins with a review of methods for estimating relative risks in the context of independent data. The context is then extended to consider clustered data. Finally, issues and controversies associated with analysing binary outcomes in the specific context of perinatal trials are discussed. 2.1 Independent Data Methods for Estimating Relative Risks The most direct way to estimate a relative risk from independent data is to use the log binomial regression model (1.2). As discussed in Section 1.3, the model is prone to convergence problems. This has prompted a number of authors to propose alternative methods for estimating relative risks, which will now be reviewed. Attention is restricted to methods which allow for adjustment of both categorical and continuous covariates, since both types of covariates are often controlled for in RCTs. This excludes from consideration the Mantel-Haenszel procedure, for example, which can be used to control for categorical but not continuous covariates [31] Constrained Estimation Methods Several of the alternative methods for estimating relative risks focus on improving the convergence of the log binomial model by constraining the estimation process. Wacholder [23] described a constrained iterative estimation procedure, where predicted probabilities, given by ˆ exp ˆ ˆ X ˆ X ˆ X i 0 1 1i 2 2i K Ki, are checked at each iteration. If any predicted probabilities exceed some maximum allowable value, such as 0.99, they are set to this maximum before parameter estimates are updated and the process iterates until convergence. This method will be referred to as constrained log binomial regression. Yu and Wang [33] suggested using the nonlinear programming procedure PROC NLP, available in SAS statistical software, to constrain the estimation process. PROC NLP can maximise a nonlinear function, subject to 8

25 constraints, using various methods of optimisation. In the context of estimating relative risks, the procedure can be used to maximise the log likelihood function for the log binomial model, subject to the constraint 0 1X1 2X 2 X 0 for all combinations of covariates 1i 2i Ki i i K Ki X, X, X contained in the dataset. This constraint is necessary to ensure that all predicted probabilities lie between zero and one COPY Method Deddens, Petersen and Lei [24] proposed the COPY method for estimating relative risks, which involves analysing a modified dataset. This dataset contains c 1 copies of the original dataset and one copy of the original dataset where the outcomes are reversed (i.e. successes become failures and vice versa). The authors suggested using c 1000 in practice. The maximum likelihood estimates in the new dataset approximate those in the original dataset but lie in the interior of the parameter space, rather than possibly lying on the boundary. The standard errors of the parameter estimates need to be multiplied by the square root of c to correct for the additional data. Rather than making physical copies of the data, appropriate weights can be applied to a single copy of the original dataset combined with a single copy of this dataset where the outcomes are reversed [27, 33, 35] Logistic Regression Methods Due to the widespread availability of software to perform logistic regression analysis and the lack of convergence problems with this model, a number of alternative approaches for estimating relative risks have been proposed based on logistic regression models. As noted in Section 1.3, the odds ratio approximates the relative risk when the outcome of interest is rare and thus, the estimated odds ratio from a logistic regression model can be used to estimate the relative risk in this case. When the outcome of interest is common, however, this approach will lead to an exaggeration of the effect of treatment [22]. This has led to the development of alternative methods based on logistic regression that do not rely on the rare outcome assumption for validity. Several authors have provided a conversion formula for calculating relative risks from odds ratios [4, 22, 36]. Given the odds ratio ( OR ) and the probability of experiencing the 9

26 outcome of interest in the control group ( 0 P ), the relative risk is given by OR 1 P OR 1 0. Zhang and Yu [22] suggested this conversion formula could be applied to confidence limits, as well as parameter estimates. This method has subsequently grown popular in health research but is not recommended, as it produces confidence intervals with coverage probability below the nominal level [29-30]. A related approach involves the calculation of standardised estimates of risk for the intervention and control groups based on a logistic regression model, with the ratio of these used to estimate the relative risk [28-29, 37-38]. Two main forms of standardisation have been considered: marginal standardisation (MS) and conditional standardisation (CS). Under MS, the risks for the intervention and control groups are calculated by averaging the predicted probabilities over a standard population assigning all subjects to the relevant treatment group. Under CS, the risks for the intervention and control groups are the predicted probabilities for subjects with a particular covariate pattern, assigning those subjects to the relevant treatment group. The covariate pattern may be chosen to be the mean of the covariates in a given population, for example. Once the relative risk has been calculated using MS or CS, appropriate confidence intervals can then be obtained via bootstrapping or the delta method [29, 38]. This approach can also be used to estimate relative risks when the data are analysed using alternative regression models for binary outcomes, such as probit, log-log or complementary log-log regression [34, 39]. According to Blizzard and Hosmer [40], this method is difficult to execute, particularly when controlling for continuous covariates. Schouten et al. [25] recognised that by manipulating the data, the logistic regression model can be used to estimate relative risks directly. Successes are duplicated to create an expanded dataset and the outcome is changed to a failure for these duplicates. If there are X successes out of N subjects in the original dataset, the probability of success in this dataset is X N, and * the probability of success in the expanded dataset is X X N 1 that * 1 *. This implies, i.e. the probability of success in the original dataset equals the odds of success in the expanded dataset. This expanded logistic regression method results in consistent estimates of the parameters in the log binomial regression model, apart from possibly the intercept. The standard errors will be incorrect and hence, robust variance estimation is 10

27 recommended [25]. This can be achieved using a generalised estimating equation approach with an independence working correlation structure [26] Log Poisson Regression Lee [3] pointed out that Cox s proportional hazards model, typically used for the analysis of survival data, can be used to estimate relative risks if the risk period is held constant. McNutt et al. [30] suggested estimating relative risks using a log Poisson regression model (i.e. a GLM with a log link and a Poisson distribution) with equal time at risk for all patients. Several authors have since noted that these two approaches are equivalent in this context [27, 31, 35]. Since the results of the Cox model are influenced by the method of handling ties and some methods will result in biased estimates of relative risk, log Poisson regression has been recommended as the preferred approach [27]. Log Poisson regression is expected to overestimate the standard errors of the parameter estimates [30]. Robust variance estimation has been suggested to correct this problem [26, 31-32] and Zou [26] termed this approach modified Poisson regression. Scaling the variance from the Poisson model by the deviance or the chi-square of the model has also been investigated but robust variance estimation is preferred [31] Log Normal Regression A nonlinear least-squares approach has been proposed for estimating relative risks, which corresponds to fitting a GLM with a log link and a normal distribution (i.e. a log normal regression model). This method will produce consistent estimates of relative risks but standard errors will be biased. Robust variance estimation or bootstrapping should be used to obtain appropriate standard errors for confidence interval calculation and statistical inference [27] Comparison of Methods There are many alternatives to log binomial regression for estimating relative risks, as described above. The relative performance of some of these methods has been examined in a number of small simulation studies. In particular, log binomial regression has been compared to constrained log binomial regression [32], SAS PROC NLP [33], the COPY method [24, 33], logistic regression with MS [29], expanded logistic regression [27, 40-41], Cox s proportional 11

28 hazards model [24, 41], log Poisson regression [26-27, 32, 35, 40] and log normal regression [27]. The key findings of these simulation studies will now be summarised. Convergence rates can be improved by utilising one of the alternatives to log binomial regression for relative risk estimation. Constrained log binomial regression resulted in improved convergence rates compared to unconstrained log binomial regression in one simulation study but convergence was still poor in some settings [32]. Constraining the estimation process using SAS PROC NLP may be more successful. Convergence rates of 100% were observed for this method in the settings studied by Yu and Wang [33], although it was implied that convergence may not always occur for this method. In another simulation study, convergence occasionally failed for the COPY method for c 100 but not for c 1000 [24]. Convergence of this method may depend on the statistical package used [40]. No convergence problems have been reported for other methods. Although constrained estimation procedures and the COPY method are not guaranteed to solve the convergence problems of log binomial regression, they do have the advantage of ensuring predicted probabilities are valid [33, 35]. In contrast, predicted probabilities greater than one have been observed for expanded logistic regression [40-41] and log Poisson regression [29, 40], and are certainly possible for log normal regression. The relative importance of producing valid predicted probabilities is a matter of debate. One point of view is that predicted probabilities should be checked for validity before making inference based on the parameter estimates from the model [40]. In particular, if the estimated risk in the treatment or control group exceeds one and hence is invalid, the estimated risk in the other group must be assumed to be incorrect in order for the estimated relative risk to be believed [35]. An opposing view is that if the effect measure is chosen for interpretability, the model used for estimation cannot be assumed to provide a perfect fit. Models with a few invalid predicted probabilities may provide a better approximation to the majority of the data than models that guarantee valid prediction [27]. Invalid predicted probabilities may be a sign of model misspecification or simply the result of sampling variability. If the expected probability for a given set of covariate values is very close to one in the population, parameter estimates based on the sample may result in a predicted probability which exceeds one [23]. 12

29 Results from two simulation studies suggest that log Poisson regression has advantages over expanded logistic regression. Blizzard and Hosmer [40] found that log Poisson regression produced valid predicted probabilities more often and generally had smaller bias and mean square error. Lumley, Kronmal and Ma [27] reported greater efficiency in parameter estimation for log Poisson regression. Other simulation results highlight some of the limitations of log Poisson regression compared to other methods. Log Poisson regression produced slightly larger standard errors and slightly lower power compared to log binomial regression replaced by the COPY method when convergence failed [35]. The relative performance of the different methods for estimating relative risks often depends on the simulation parameters used. Yu and Wang [33] found that bias was smaller for constrained estimation using SAS PROC NLP when the maximum probability of success was large, but smaller for the COPY method otherwise. Parameter estimates could be more or less efficient for log Poisson regression compared to log normal regression, depending on the setting [27]. Localio, Margolis and Berlin [29] compared bootstrapping and the delta method for calculating confidence intervals when relative risks were estimated from logistic regression using MS. Bootstrapping was preferred in terms of coverage when the sample size was small but the methods performed similarly when the sample size increased Specific Aim 1 The literature indicates that although comparisons have been made previously between methods for estimating relative risks, only a few of the many available methods for calculating relative risks have been considered in each case. Further, the results of these studies do not provide a clear indication regarding which method(s) are preferable to use in practice. The first specific aim of this thesis is to compare the performance of methods for estimating relative risks in the context of RCTs with independent observations, and to make recommendations about which method(s) should be used in practice. I address this aim in Chapter Clustered Data The methods for estimating relative risks described in Section were all proposed in the context of independent data. In this section, the focus is on clustered data and three main 13

30 areas are covered. First, general issues involved with analysing clustered data are discussed. These issues are relevant to the extension of any method for analysing independent data to handle clustered data. Second, methods for simulating clustered binary outcomes are reviewed. A simulation method must be chosen to allow different approaches for estimating relative risks based on clustered data to be compared in a finite sample size setting. Finally, extensions of the methods for estimating relative risks for independent data to the clustered data setting are considered. Following the notation used previously, subscripts i and j will now be used to represent clusters and units within clusters respectively ( i 1,, m ; j 1,, ni ). For instance, a binary for unit j in cluster i is represented by Y ij, with corresponding probability ij of experiencing the outcome of interest. The indicator variable for treatment group is represented by X 1ij, where treatment may be allocated at the cluster ( X1 ij X1 i ) or unit level. Random cluster effects are represented by a i Analysing Clustered Data Clustered data are characterised by the dependence between observations collected on units belonging to the same cluster. The degree of dependence is typically measured by the intraclass or intracluster correlation coefficient (ICC) [42]. Clustered data are common in RCTs, and may result from repeated measurements taken on the same subject over time (longitudinal data), or measurements taken at the same time on sub-units within the primary unit (for example, eyes within subjects or patients within hospitals). In this thesis, only the latter form of clustering is considered. When clustering is present in the data, it should be taken into account in the statistical model. Ignoring the clustering can lead to substantially biased standard errors of parameter estimates, potentially resulting in false conclusions about the effect of predictors on the outcome of interest [43]. Two common extensions to GLMs to account for clustering in the analysis are MEMs and GEEs. These were introduced in Section 1.1 and will now be discussed in detail. 14

31 MEMs versus GEEs Although MEMs and GEEs both account for clustering, the two approaches differ in a number of important ways. First, MEMs and GEEs deal with clustering differently. MEMs account for clustering explicitly by incorporating a random cluster effect in the model. This is commonly referred to as a conditional or cluster-specific approach. In contrast, GEEs use a marginal or population-averaged approach, controlling for clustering implicitly through the use of a working correlation structure [17, 44]. Second, different assumptions are required when fitting MEMs and GEEs. For MEMs, a distribution must be assumed for the random cluster effects which may be difficult to verify. Misspecification of the random effects distribution can have a substantial impact on the results [45]. For GEEs, no assumptions about the nature of the clustering are necessary. In fact, GEEs will produce consistent parameter estimates even if the working correlation matrix is misspecified, provided the mean model is correctly specified [16]. Third, the approaches differ in terms of the type of missing data that can be accommodated. GEEs require any missing data to be missing completely at random for consistency. By comparison, MEMs are subject to less stringent missingness assumptions, only requiring any missing data to be missing at random [46]. This distinction may not be an issue in practice, as multiple imputation is commonly used to fill in missing data in RCTs prior to analysis. Finally, parameters from MEMs and GEEs have different interpretations. MEMs estimate the clusterspecific effect, or the effect of covariates on the outcome conditional on the value of the random effect. These models are therefore suited to estimating the effect of covariates which vary within clusters [17]. Using these models to estimate the effect of covariates which remain constant within clusters, such as treatment in cluster RCTs, is controversial, however. Cluster-specific estimates of the effect of cluster-constant covariates describe the unobserved effect of changing the covariate within the cluster, and may be viewed as a form of model extrapolation [47]. The effect of cluster-constant covariates may be better estimated using GEEs which measure the average effect of the covariate on the outcome (i.e. the population-averaged effect). The use of GEEs for estimating the effect of cluster-varying covariates has been criticised though, since they do not make use of comparisons that may be made within clusters [17]. This suggests that the choice between fitting MEMs or GEEs may be influenced by whether the covariate of interest is fixed or varying within clusters, and will therefore be determined by the context, with neither approach clearly favourable in all situations. 15

32 Given that parameters of MEMs and GEEs have different interpretations, the choice between them is often important. However, several special cases have been identified where the parameters coincide and hence both a cluster-specific and a population-averaged interpretation may be applied, whichever type of model is used for estimation. Consider a MEM of the form g a X β a, where g is the link function and other variables are as defined previously. ij i ij i The cluster-specific and population-averaged parameters coincide if g is the identity link. Further, the parameters coincide apart from the intercept if g is the log link. For other link functions, such as the logit link, the parameters generally differ [18]. This means that if the relative risk is of interest and estimation is based on a model with a log link, then the choice between MEMs and GEEs should not be influenced by the desired parameter interpretation. In contrast, interpretation should be carefully considered when estimating odds ratios using a model with a logit link, since marginal and conditional odds ratios differ [17] Choosing an Estimation Method for MEMs If an analysis is to be performed using MEMs, an approximate method of estimation must be chosen. This is necessary to avoid the computational difficulties associated with integrating out the random effects from the likelihood function. A popular estimation approach involves approximating the nonlinear model by a linear model using a first-order Taylor series expansion and then applying quasi-likelihood estimation. If the expansion is carried out around the current estimate of the fixed part of the predictor, the method is referred to as marginal quasi-likelihood (MQL) estimation. The alternative is to expand around the current estimates of both the fixed and random parts of the predictor, a process referred to as penalised quasi-likelihood (PQL) estimation [48]. Both PQL and MQL estimation have been shown to be biased for binary outcomes when the number of units per cluster is small [48-49]. PQL is preferred to MQL in this case, but may not be possible in practice due to potential convergence problems with this estimation method [50]. Bias may be reduced by using a second-order Taylor series expansion to approximate the nonlinear model, particularly for PQL estimation [48]. Alternatively, an estimation method which relies on numerical integration can be used, such as adaptive Gauss- Hermite quadrature (AGHQ) [51]. 16