Proper analysis in clinical trials: how to report and adjust for missing outcome data

Transcription

1 DOI: / Commentary Proper analysis in clinical trials: how to report and adjust for missing outcome data M Joshi, a, * A Royuela, b,c J Zamora b,c a Centre for Primary Care and Public Health, Barts and The London School of Medicine and Dentistry, London, UK b CIBER Epidemiologıa y Salud Publica (CIBERESP), Madrid, Spain c Clinical Biostatistics Unit, Hospital Ramon y Cajal, IRICYS, Madrid, Spain Correspondence: Dr M Joshi, Centre for Primary Care and Public Health, Barts and The London School of Medicine and Dentistry, Yvonne Carter Building, 58 Turner Street, London E1 2AB, UK. m.a.i.joshi@qmul.ac.uk Accepted 16 February Please cite this paper as: Joshi M, Royuela A, Zamora J. Proper analysis in clinical trials: how to report and adjust for missing outcome data. BJOG 2013;120: Missing data is a problem that occurs widely in medical research, and indeed is difficult to avoid. The aim of this commentary is to deal with its implications in clinical trials. In analysing the results of a randomised controlled trial, it is important to ensure that the main benefit of randomisation has not been compromised: namely, that the treatment arms (experimental and control) remain comparable in everything except the interventions being compared. 1 Missing outcome data is a serious problem because of its ability to bias the findings of a clinical trial. The bias affects the direction of the effect observed, but data loss can also affect the precision, making the results unreliable. A good definition of missing data is given by Little, values that are not available and that would be meaningful for analysis if they were observed. 2 As our interest is in trials, we concentrate on missing outcome data, although methods have been developed to handle missing covariates as well. The bias because of missing data may depend on the reason why data are missing. We therefore have to consider the mechanism of how data comes to be lost. Little and Rubin defined three patterns of missing data. 3 Table 1. Types of missing data patterns and implications for clinical trial analysis MCAR Missing completely at random MAR Missing (conditionally) at random MNAR Missing not at random Definition* Example: a study about the intensity of pelvic pain and related factors. The data is collected by questionnaires given to women at a clinic The probability of a particular value being missing is completely independent of both the observed data and the unobserved data Some questionnaires are randomly lost by accident or random errors in entering data The probability of a particular value being missing depends only on the observed data Disabled women experience difficulty in attending the clinics where the questionnnaires are administered. Women s disability information is recorded Bias None None, because missing data do not depend on the unobserved data Implications for the analysis in clinical trials Complete case analysis. Loss of power, imprecision. Assumption cannot be tested. Required for most types of analyses (e.g. Multiple Imputation). Assumption can be tested The probability of a particular value being missing depends on the unobserved data Some of the disabled women are depressed and less likely to attend, but this information has not been recorded Yes, because missing data depend on unseen observations It is very rare to know the appropriate model for this data loss mechanism *White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med 2011;30: ª 2013 The Authors BJOG An International Journal of Obstetrics and Gynaecology ª 2013 RCOG 915

2 Joshi et al. 1 Missing completely at random (MCAR), in which the available data is just as representative of the population from which it were taken as the complete data in the sample. 2 Missing at random (MAR), in which, given the values that we have, the missing data does not depend on the unobserved data. 3 Missing not at random (MNAR), in which the probability of missing data depends on the unobserved data. This is the standard nomenclature. To avoid misunderstanding, we suggest that conditionally at random (which Rubin terms MAR) might be less misleading than at random, which could be interpreted to mean completely at random. As an example of these patterns, consider questionnaires administered at an ambulatory clinic for women experiencing pelvic pain. If some questionnaires are randomly lost, the pattern of loss is MCAR. If women who are disabled are transported directly to hospital avoiding the ambulatory clinics where pelvic pain is being measured, such that data cannot be obtained from some of them, and the probability of non-attendance is predictable from the data we have, the pattern is MAR. However, the reason for missing data may lie outside the clinic, and so depend on unobserved variables. This might occur, for example if some of the disabled women are depressed and less likely to attend, but this information has not been recorded. Then the pattern is MNAR. Table 1 summarises the patterns of missing data. The Consolidated Standards of Reporting Trials (CON- SORT) statement was designed to improve the quality of reporting of randomised controlled trials. 4 It comprises an evidence-based set of recommendations for reporting RCTs that includes a 25-item checklist and a flow diagram. The checklist deals with the design of the trial, the analysis and interpretation of the results. The flow diagram displays the progress of participants through the stages of enrolment in the trial, intervention, allocation, follow-up and data analysis. A flowchart enables us to keep track of missing data at each stage of the trial, and so makes clear the problem that we will have to deal with in the analysis. People whose Assessed for eligibility (n = 138) Enrolment Refused to participate (n = 35) Randomisation error (n = 3) Randomized (n = 100) Allocation Conventional surgery Allocated to intervention (n = 51) Vessel sealing Allocated to intervention (n = 49) Follow-up Lost to follow up: 6 weeks: n = 13 6 months: n = 16 Lost to follow up: 6 weeks: n = 11 6 months: n = 9 Analysis Analysed (n = 35) Analysed (n = 40) Figure 1. Example of the CONSORT flow of participants through each stage of the trial (Lakeman et al. BJOG 2012;119: ). 916 ª 2013 The Authors BJOG An International Journal of Obstetrics and Gynaecology ª 2013 RCOG

3 Missing data in randomised controlled trials (RCTs) participation ceased after allocation are unlikely to be representative of all participants in the study. Knowing the number of participants who did not receive the intervention as allocated, or did not complete treatment, enables the reader to assess to what extent the estimated effect of therapy might be biased. Therefore, our first recommendation is to present a participant flowchart following the CONSORT statement. In Figure 1 we show an example of a clinical trial study flowchart published in BJOG recently. Such a flowchart should give a more comprehensive picture of the missing data than a single summary statistic. The intention to treat (ITT) principle means comparing patients in the groups to which they were originally randomly assigned. 5 This is generally interpreted as including all patients, regardless of whether they actually satisfied the entry criteria, the treatment was actually received, and whether they subsequently withdrew or deviated from the protocol. This maintains the comparability of the groups apart from random variation, which is the reason for randomization. As White et al. 6 point out, it is not clear how the principle can be applied when outcome data are missing. It is not clear whether any loss of participants can be attributed to chance, an improvement in their condition, a worsening in their condition or to side effects, or can be attributed to any other characteristic of the participant. The traditional way to deal with missing data is complete case analysis, in which patients with missing data are not included. If the loss mechanism is MCAR, it is a sensible method. MCAR is a strong assumption, however. Under MCAR, the results are unbiased, although the statistical power may be reduced. Our second recommendation is about performing a main analysis of all observed data that are valid under a plausible assumption about the missing data. Other more complex methods for dealing with missing data are based on the MCAR or MAR assumptions, such as weighting and imputation procedures. Weighting procedures consist of weighting every observed value by the inverse of its probability of being observed, given the covariates. 7 Imputation methods include single and multiple imputation. The first implies filling in or imputing the missing values in the data set. In multiple imputation, missing values are imputed using a set of sampled values based on models for the missing data conditional on all relevant observed data, and later on appropriately combining results obtained from each of them. 7,8 Table 2. Hypothetical scenarios for missing data and the results of various methods to deal with data loss Good outcome Bad outcome Missing data Total Proportion good (a) Actual results with complete case analysis Experimental Control Good outcome Bad outcome Total Proportion good (b) Cases with missing data had good outcomes Experimental Control (c) Cases with missing (control) data had good outcomes; cases with missing (experimental) data had bad outcomes Experimental Control (d) Cases with missing (experimental) data had good outcomes; cases with missing (control) data had bad outcomes Experimental Control (e) Cases with data missing completely at random Experimental Control (f) Cases with data missing not at random Experimental 49 + x 21 + (30 x) x Control 35 + y 35 + (30 y) y In (a) we have the same proportion of good results as in (e), i.e. the estimate is unbiased; however, because the effective sample is smaller, the confidence interval for it will be about 11% greater. As an example, if x = 10 but y = 20 (because depression is less well controlled in the controls and causes non-participation), the proportions in the farthest right column for experimental and control subjects will be 0.64 and 0.55, a less convincing result than in (a). ª 2013 The Authors BJOG An International Journal of Obstetrics and Gynaecology ª 2013 RCOG 917

4 Joshi et al. Our assumptions about the mechanism of loss, however (typically, MAR), might be mistaken. We need to find out whether the results are robust to the type of assumptions we make. To this end, we may vary the assumption about randomness and see whether the findings are robust to such variation. White suggests that if the main analysis assumes similarity between groups who are and are not lost to follow-up, a good sensitivity analysis might assume that the group who are lost to follow-up have systematically worse outcomes. 6 With binary outcomes, look at the bestand worst-case scenarios. Table 2 contains a hypothetical example to illustrate the necessity of taking missing data into account in analysis. As can be seen, different scenarios of missing data can make a big difference to the result. We therefore make a third recommendation, namely a sensitivity analysis. The results of the sensitivity study should be reported by a statement in the published study about the robustness of the findings to changes in the assumptions. Sterne et al. 9 offer guidelines for the reporting of analyses potentially affected by missing data, which are consistent with our framework. Such information could be included in an appendix to a paper. They recommend reporting the number of missing values for the variables of interest, or the number of cases with complete data for each important component of the analysis. They recommend giving reasons for missing data, particularly in terms of other variables, and describing any important differences between individuals with complete and incomplete data. Where applicable, these could support the assumptions upon which missing data are handled when performing ITT, as well as the sensitivity analysis afterwards. In the latter connection they recommend giving details of the modelling used in multiple imputation. Whereas our discussion above is mainly about reporting, it is important to mention the design stage of a study. When doing a sample size calculation it may be worthwhile to anticipate drop-outs by increasing the recommended sample size by an appropriate amount. The inflation figure may be suggested by experience with research in related areas. In the conduct of the study it is essential to ensure that all efforts are made to minimise losses. We made a brief search of the BJOG website for recent articles in BJOG that mentioned missing data explicitly by using this expression in the search field. This yielded five articles published in the current millennium, albeit not connected with clinical trials. Of these, the three earlier ones mentioned the problem, but did not attempt a formal statistical treatment, whereas the two more recent ones did so. 13,14 Notwithstanding the small number, if they represent the wider picture, the trend is welcome, certainly for reporting the results of clinical trials. Our recommendations are summarised in Box 1. Box 1 Summary of recommendations Plan sample size, taking losses to follow-up into consideration Inflate the sample size by taking into account the potential for losses and take all measures to avoid missing data during the study. Include a participant flowchart that shows all losses Information about the flow of participants enables missing data to be identified. Intention-to-treat analysis adjusted for missing data should be the primary analysis The basis should include reasonable assumptions about missing data; complete case analysis fails to meet the intention to treat principle. Include a sensitivity analysis See whether the results are robust to different assumptions. Disclosure of interests None to declare. Contribution to authorship All authors actively participated in the preparation of this article and agreed on the order shown. All shared equally in the conception and planning. MJ wrote the initial drafts of the main text and covering letter (and the revised versions), which were then sent to the co-authors (AR and JZ), and drafts were extensively revised in light of correspondence between the three authors. AR was responsible for the initial draft of Table 1. Details of ethics approval Not required. Funding None. Acknowledgement The initiative came from Prof K Khan, who provided helpful advice and feedback on an earlier draft. & References 1 O Neill RT, Temple R. The prevention and treatment of missing data in clinical trials: an FDA perspective on the importance of dealing with it. Clin Pharmacol Ther 2012;91: Little RJ, D Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, et al. The prevention and treatment of missing data in clinical trials. N Engl J Med 2012;367: Little RJ, Rubin D. Statistical Analysis with Missing Data. Hoboken, NJ: Wiley, Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Int J Surg 2011;9: ª 2013 The Authors BJOG An International Journal of Obstetrics and Gynaecology ª 2013 RCOG

5 Missing data in randomised controlled trials (RCTs) 5 Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ 1999;319: White IR, Horton NJ, Carpenter J, Pocock SJ. Strategy for intention to treat analysis in randomised trials with missing outcome data. BMJ 2011;342:d40. 7 Carpenter J, Kenward M. Missing data in clinical trials a practical guide. Birmingham, UK: National Institute for Health Research; Ref Type: Report. 8 White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med 2011;30: Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ 2009;338:b Hiller L, Radley S, Mann CH, Radley SC, Begum G, Pretlove SJ, et al. Development and validation of a questionnaire for the assessment of bowel and lower urinary tract symptoms in women. BJOG 2002;109: Withagen MI, Wallenburg HC, Steegers EA, Hop WC, Visser W. Morbidity and development in childhood of infants born after temporising treatment of early onset pre-eclampsia. BJOG 2005; 112: Booth SJ, Pickles MD, Turnbull LW. In vivo magnetic resonance spectroscopy of gynaecological tumours at 3.0 Tesla. BJOG 2009;116: Lier D, Ross S, Tang S, Robert M, Jacobs P. Trans-obturator tape compared with tension-free vaginal tape in the surgical treatment of stress urinary incontinence: a cost utility analysis. BJOG 2011;118: Flach C, Leese M, Heron J, Evans J, Feder G, Sharp D, et al. Antenatal domestic violence, maternal mental health and subsequent child behaviour: a cohort study. BJOG 2011;118: ª 2013 The Authors BJOG An International Journal of Obstetrics and Gynaecology ª 2013 RCOG 919