Clinical research in AKI Timing of initiation of dialysis in AKI

Clinical research in AKI Timing of initiation of dialysis in AKI Josée Bouchard, MD Krescent Workshop December 10 th, 2011 1

Acute kidney injury in ICU 15 25% of critically ill patients experience AKI AKI is associated with increased mortality and morbidity <5% of ICU patients will require dialysis In patients requiring dialysis, mortality 50 70% Timing of initiation of dialysis = top priority in clinical research (AKIN) 2

Clinical research project Study on timing of initiation of dialysis in AKI Critically ill adult patients with AKI Confusion on AKI definition and stages For simplicity, use of RIFLE/AKIN classifications to define timing Early defined by R or I / 1 or 2 Late defined by F / 3 IHD or CRRT dose according to recent large RCTs Mortality and dialysis independence as outcomes 3

Clinical research topics Bias and confounding factors Meta analysis 4

Validity External validity (generalizability) Internal validity Random errors (chance) Systematic errors (bias) Random errors: difference in patients population Mehta s trial on dialysis modality 5

Bias Main types of bias Selection incidence prevalence bias (survival bias) loss to follow up bias publication bias (small study effects) ascertainment bias (sample bias) Information misclassification bias lead time bias Confounding factors Selection bias: when a systematic error in the enrollment of individuals in a study determines a biased association between exposure and outcome. Information bias: occurs during data collection. Confounding: which some authors consider also as a kind of bias not an error in the study but a phenomenon identified in a study and must be understood 6

Loss to follow up bias Individuals lost to follow up do not have the same probability of having the outcome of interest vs. individuals who remain under observation Should be <20% to minimize bias Kidney failure = no symptoms, so less likely to seek FU In the recent ATN RCT trial on dialysis dose which included >1000 patients, no more than 10 patients per group had missing data on renal recovery at 28 days 7

Publication bias (small study effects) A Tripepi, Kidney Int 2008 Published studies should constitute an unbiased sample of all studies performed. When these assumptions are not met, a literature review will give a distorted view of the exposure outcome association of interest The simplest and most commonly used method to detect publication bias is the funnel plot A funnel plot is a graph where the effect of a given treatment of each trial is plotted against some measure of its size, such as the precision, the standard error, or the overall sample size. These plots are referred to as funnel plots because they should be shaped like a funnel if no publication bias is present. This shape is expected because the estimate of the effect of a treatment has a larger variability in smaller studies. Since smaller and negative studies are less likely to be published, trials in the bottom left hand corner of the graph are often omitted, creating a degree of asymmetry in the funnel. Funnel plot uses a scatter plot of studies that relates the magnitude of the treatment effect to the weight of the study. An inverted funnel shaped, symmetrical appearance of dots suggests that no study has been left out, whereas an asymmetrical appearance of dots, typically in favour of positive outcomes, suggests the presence of publication bias. Controversies remain about including unpublished studies, their omission increases the chances that studies with positive results will be overrepresented in the review (leading to a systematic overestimation of the treatment effect, referred to as publication bias) The tendency for authors to differentially submit and journals to differentially accept studies with positive results constitutes a serious threat to the validity of systematic reviews. Reviews based on a small number of small studies with weakly positive effects are the most susceptible to publication bias. 8

Ascertainment bias (sample bias) When a sample is collected in such a way that some individuals of the intended population are less likely to be included than others If not accounted for, results can be erroneously attributed to the phenomenon under study rather than to the method of sampling Example: creatinine measurements in a database for CKD creatinine is an inadequate marker for GFR, even in the steady state. An increase in creatinine level is not a precise quantitative reflection of decrements in GFR. Comparisons between individuals are, therefore, confounded, even when adjustments for the baseline level of creatinine are made. When using an absolute increase in serum creatinine level as an indicator of an adverse effect on the kidney, a greater degree of injury is needed in individuals with higher baseline GFRs to achieve the definition of AKI (Table 3). There is, therefore, a bias towards AKI being identified more easily in those with lower base line GFRs or higher baseline serum creatinine concentrations (i.e. worsening degrees of chronic kidney disease). Could this ascertainment bias contribute to the higher incidence of AKI related to CIN, for example, that is reported in patients with impaired baseline kidney function? This ascertainment bias is reduced (but not completely eliminated) when a relative increase in serum creatinine level is used to define AKI 9

Ascertainment bias (sample bias) Solomon, Nat Clin Pract 2008 a bias towards AKI being identified more easily in those with lower baseline GFRs or higher baseline serum creatinine concentrations (i.e. worsening degrees of chronic kidney disease). Could this ascertainment bias contribute to the higher incidence of AKI related to CIN, for example, that is reported in patients with impaired baseline kidney function? 10

Misclassification bias Recall bias Interviewer bias Observer bias Regression dilution bias AKI litterature: Dialysis doses studies Tripepi, Kidney Int 2008 Misclassification bias originates when the process to detect the exposure status (exposure identification bias) and/or the outcome (outcome identification bias) is imperfect, that is, exposed/diseased individuals are classified as nonexposed/nondiseased and vice versa. Observer bias: Knowledge of exposure status by the outcome assessor may influence the assessment process and therefore produce biased results. Regression dilution bias: The frequency of regression to the mean (defined here as the decrease of the association reported in the first published paper with later studies on the same topic appear) can also be approached by systematic reviews of meta analyses. This fact is related to publication bias: if an article with significant results is published it may trigger that other, remaining in researchers drawers, to appear claiming that they do not find such association. It can be also attributable to a refinement of methods with time: a better methodology reduces bias and shows a trend to a lower strength of association. 11

Bias Bias can be prevented at two levels: 1. By choosing the appropriate study design for addressing the study hypothesis 2. By carefully establishing the procedures of data handling and the definitions of exposures and outcomes 12

Confounding factors A confounding factor: 1. Should be a risk factor for the disease 2. Must be associated with the exposure (unequally distributed between exposure groups) 3. Must not be an effect of the exposure (not be part of the causal pathway) When an investigator tries to determine the effect of an exposure on the occurrence of a disease (or other outcome), but then actually measures the effect of another factor, a confounding variable. 13

Confounding factors Jager, Kidney Int 2008 14

Confounding factors Jager, Kidney Int 2008 The analysis of study data itself will not indicate what is or is not a confounder. Only a considerable knowledge of pathophysiological mechanisms and potential causal pathways will assist an investigator to decide whether a variable satisfies the criteria for being a potential confounder or not. 15

Confounding factors Techniques are described that may be applied to prevent or control for confounding: BEFORE STUDY: randomization restriction matching AFTER STUDY: stratification multivariate analysis Randomization helps to prevent selection bias by the clinician (sometimes also referred to as confounding by indication ). Although the balance of known and unknown confounders may be incomplete, the randomization process does guarantee that any differences between the two groups are due to chance and not due to the choice of the physician Restriction: example for age, only below 65 yo. Matching often used for case control studies Best evidence = RCTs and meta analysis of RCTs. Bias in RCT most often from lack of concealment of randomization, lack of blinding and failure to report reasons for excluding patients 16

Meta analysis Combine results of several independent studies on a specific topic Purposes: 1. Increase stat power by increasing sample size 2. Resolve uncertainty when reports do not agree 3. Improve estimates of effect size in the absence of large, definitive clinical trials, meta analyses can provide important information to guide patient care as well as future clinical research. by combining the samples of the individual studies, the size of the overall sample is increased, enhancing the statistical power of the analysis and reducing the size of the confidence interval for the point estimate of the effect. Increases generalizability but also increases heterogeneity. Effect size: degree to which the phenomenon is present in the population 17

Bhandari, J Can Surg 2004 18

Reporting guidelines Meta analysis PRISMA (formerly QUOROM) PRISMA checklist and flow diagram 19

Meta analysis Small trials more prone to publication bias Low methodological quality trials have an increased efficacy of treatment compared with high quality trials Heterogeneity more present in observational studies vs. RCTs since less control of bias 20

Meta analysis Meta analysis weight studies according to: Size (size of the square) Also consider: Quality Type of studies 6 methodologic domains: randomization and blinding, population, intervention, outcomes, follow up and statistical analysis 21

Meta analysis Zlowodzki, Acta Orthopedica 2007 3 criteria to consider when deciding whether the results are sufficiently similar to warrant a single estimate of treatment effect (AGGREGATE SUMMARY ESTIMATE = diamond shape) First, point estimates, second CI and third homogeneity: When the p value associated with the test of homogeneity is small (e.g., < 0.05), chance becomes an unlikely explanation for the observed differences in the size of the effect. Unfortunately, a higher p value (0.1 or even 0.3) does not necessarily rule out important heterogeneity. The reason is that when the number of studies and their sample sizes are small, the test of heterogeneity is not very powerful. Hence, large differences between the apparent magnitude of the treatment effect among the primary studies (i.e., the point estimates) dictate caution in interpreting the overall findings, even in the face of a non significant test of homogeneity. Conversely, if the differences in results across studies are not clinically important, then heterogeneity is of little concern, even if it is statistically significant. 22

Meta analysis Zlowodzki, Acta Orthopedica 2007 pooling of study results seems justified by the nonsignificant tests of heterogeneity, reasonable similarity of results (point estimates) and widely overlapping CIs around those point estimates. I2 values, range from 0% to 100%, describe the proportion of variation in treatment effect estimates that is due to genuine variation rather than sampling error (22). 0% indicates no observed heterogeneity. I2 values of 25%, 50%, and 75% as low, moderate, and high, respectively. 23

Heterogeneity tests Meta analysis describe the proportion of variation in treatment effect estimates that is due to genuine variation rather than sampling error not very powerful different methods Cochran s Q statistic I 2 statistic: ranges from 0 100% generalised Q When Q is larger than its expected value E[Q] under the null hypothesis of no heterogeneity, the difference Q E[Q] can be used to furnish the most popular estimate of the heterogeneity parameter, using the DerSimonian and Laird method Higgins and Thompson s I2 statistic [4,5] is also a simple function of Q and quantifies the proportion of total variation that is between trial heterogeneity. approximates the per centage of total variation (within and between studies) due to between study variation. Unlike Q, I2 is designed to be independent of the number of trials constituting the meta analysis and independent of the outcome s scale, so it can easily be compared across meta analyses. It is now reported as standard, with or without Cochran s Q. Explaining heterogeneity via the pre specification of trial subgroups, graphical diagnostic tools and sensitivity analyses produced a more desirable outcome than an automatic application of the random effects model. Generalised Q statistic methods for quantifying and adjusting for heterogeneity should be incorporated as standard into statistical software. 24

Meta analysis Fixed effect vs. random effects metaregression The fixed effects regression model does not allow for between study variation, this also yields in to significant results too easy The random or mixed effects model allows for within study variation and between study variation Choice based on heterogeneity Which model to choose The simple regression model does not allow for within study variation, this yields in to significant results too easy. The fixed effects regression model does not allow for between study variation, this also yields in to significant results too easy. The random or mixed effects model allows for within study variation and between study variation and is therefor the most appropriate model to choose. Whether there is between study variation can be tested by testing whether the effect sizes are homogeneous. If the test shows that the effect sizes are not heterogeneous the fixed effects meta regression might seem appropriate, however this test often does not have enough power to detect between study variation. Besides the lack of power of this test, you can reason that the fixed effects assumption of homogeneous effect sizes is rather weak, because it assumes that all studies are exactly the same. However you can assume that no two studies are exactly the same. To cope with the fact that each study is different (different sample; different time; different place; etc) a random or mixed effects model is always the appropriate model to choose and gives the most reliable results. Fixed effect assumes that the treatment effect is the same for every study (not good if heterogeneity) and random effects assumes that an effect may vary across studies because of the differences between studies (more conservative, wider CI) ie. Choice of model based on heterogeneity Some say that random should always be used but limitation in taking covariates into account (different study populations or cotreatments to explain heterogeneity Fixed effect models estimate the weighted mean of the study estimates, whereas random effects models estimate the mean of a distribution from which the study estimates were sampled. 25

Meta analysis Explanations of heterogeneity: Sensitivity analyses (subgroup analyses) based on a priori hypotheses Clinical aspects Methodological aspects Specified subgroups by patients and trial characteristics 26

Conclusion Studies on timing of initiation of dialysis in AKI offer a great challenge in clinical research Bias and confounding factors need to be identified and minimized Critically appraise a meta analysis can help to design future studies in this field 27

References Bhandari M, Devereaux PJ, Montori V, Cina C, Tandan V, Guyatt GH. Users guide to the surgical literature: how to use a systematic review and meta analysis. J Can Surg 2004; 47: 60 7 Delgado Rodriguez M. Systematic reviews of meta analyses: applications and limitations. JECH 2006;60: 90 2 Garg AX, Hackam D, Tonelli M. Systematic review and metaanalysis: when one study is just not enough. cjasn 2008; 3: 253 60 Jager KJ, Zoccali C, MacLeod A, Dekker FW. Confounding: what it is and how to deal with it. Kidney Int 2008; 73: 256 60 Tripepi G, Jager KJ, Dekker FW, Wanner C, Zoccali C. Bias in clinical research. Kidney Int. 2008; 73. 148 153 Zlowodzki M, Poolman RW, Kerkhoffs GM, Tornetta P, Bhandari M. How to interpret a meta analysis and judge its value as a guide for clinical practice. Acta Orthopaedica 2007; 78: 598 609 28