Assessment of reproducibility of creatinine measurement and MELD scoring in four liver transplant units in the UK

Transcription

1 Nephrol Dial Transplant (2010) 25: doi: /ndt/gfp556 Advance Access publication 5 November 2009 Assessment of reproducibility of creatinine measurement and MELD scoring in four liver transplant units in the UK Carol Goulding 1, Evangelous Cholongitas 1, Devi Nair 1, Andrew Kerry 1, David Patch 1, Murat Akyol 2, Simon Walker 2, Derek Manas 3, David Mc Clure 3, Liesl Smith 3, Neville Jamieson 4, Ingela Oberg 4, David Cartwright 4 and Andrew K. Burroughs 1 1 The Liver Unit, The Royal Free Hospital, London, UK, 2 The Liver Unit, Royal Infirmary of Edinburgh, Edinburgh, UK, 3 The Liver Unit, The Newcastle upon Tyne NHS Hospital Trust, Newcastle upon Tyne, UK and 4 The Liver Unit, Addenbrookes Hospital, Cambridge, UK Correspondence and offprint requests to: Andrew K Burroughs; Andrew.burroughs@royalfree.nhs.uk Abstract The Model for End-Stage Liver Disease (MELD) or similar scoring system is proposed in the UK for prioritization for liver transplantation. We evaluated the reproducibility of creatinine measurements and therefore MELD scores in four liver transplant units in the UK. Methods. All transplant units were invited to participate; four agreed to do so, contributing 36 patients awaiting liver transplantation. Blood was collected from these 36 and divided into aliquots then sent to the four participating centres. Every centre measured creatinine and bilirubin for every patient. The results were analysed for the degree of agreement between centres for creatinine and MELD scores using the Bland Altman method and Wilcoxon rank test. Results. The mean creatinine value varied from 101 μmol/ l in centre C to 110 μmol/l for the same sample in centre A, with significant differences between the four centres (P < 0.05, Wilcoxon signed-rank test). The MELD scores were significantly different between centre C and all other centres (P < 0.05). Conclusion. This study demonstrates lack of agreement in measurement of serum creatinine and MELD scoring between four UK transplant centres. A difference in two MELD points can have a significant impact on patient outcome, and these factors will have to be addressed if a UKwide transplant list is to be initiated. Keywords: creatinine; liver transplant; MELD score; organ allocation Introduction The continuing shortage of organ donors means that patients are still dying whilst on the waiting list for liver transplantation. In February 2002, the Model for End-Stage Liver Disease (MELD) scoring system was implemented in the USA in an effort to produce an organ allocation system which had justice and equity, maximal utility and transparency as its priorities [1]. It had the advantage of removing waiting time on the transplant list as an influence on organ allocation, as this did not correlate with disease severity and often meant that when a donor liver became available it was not necessarily used for the sickest first. The MELD score was considered to be objective, comprising international normalized ratio (INR), creatinine and bilirubin, removing the more subjective assessment of ascites and encephalopathy that are included in the Childs Turcotte Pugh (CTP) score. The ability of MELD to predict 3-month mortality on the liver transplant waiting list has been shown by several studies to be approximately 80% accurate, i.e. to have a c-statistic (i.e. the concordance statistic, an area under the receiver operator curve) of 0.8 [2,3]. This was similar and not superior to CTP in a recent literature review [4]. Initial results following introduction of the MELD system in the USA showed a reduction in waiting list deaths and increased transplantation rates for those on the list, in all areas in the USA and across all diagnoses, blood groups, gender and race [5]. From February 2002 to February 2003, i.e. the first year analysed since MELD was introduced, there was a 3.5% reduction in waiting list mortality [6]. Merion et al. have shown that the survival benefit from transplant increases with increasing MELD score, with no real benefit seen in those with a MELD score <18; in fact, those with a MELD score <15 had a higher mortality following a transplant than whilst remaining on the waiting list [7]. Pre-transplant mortality risk was seen to rise exponentially rather than linearly with change in MELD score; thus, near the higher end of the MELD score, variations in one or two points are very significant clinically. Studies from Europe have also shown the MELD scoring system to be an effective method of organ allocation [8,9], and recently the MELD system has been adopted by Eurotransplant. However, in a recent article by Ravaioli et al. looking at the impact of the MELD system on two transplant units in Italy using the same organ procurement zone, it was noted that one unit had a significantly higher risk of patients being too sick and therefore being removed from the list than the other (P < 0.01). This was discovered to be due to the fact that both labs had different ranges for normal cre- The Author Published by Oxford University Press on behalf of ERA-EDTA. All rights reserved. For Permissions, please journals.permissions@oxfordjournals.org

2 Creatinine measurement and MELD scoring in liver transplant units 961 Table 1. Laboratory techniques and normal ranges Centre Creatinine lab technique Creatinine μmol/l normal range Bilirubin lab technique Bilirubin μmol/l normal range A Kinetic Jaffe O'Leary method 3 16 B Kinetic Jaffe (males) O'Leary method (females) C Dade dimension Dade dimension 0 17 D Kinetic Jaffe (males) Roche modular P unit (females) atinine despite similar methodology in measurement. Therefore, the patients in the first lab had their creatinine underestimated when compared to the second lab and therefore had lower MELD scores. When both labs normalized to the same creatinine range, the difference in waiting list dropouts disappeared [10]. Many other national transplant organizations are evaluating scores and prognostic models to differentiate severity of liver disease in those waiting for liver transplantation. The MELD scoring system for organ allocation was adopted by the United Network for Organ Sharing (UNOS) because they felt it was simple, objective and verifiable and yielded consistent results. However, the underlying assumption that measurement of the components of the various scores, and in particular the measurement of INR and creatinine, is standardized across different laboratories and countries has been shown to be an erroneous assumption. It has been shown that these measurements are not accurately reproducible or transportable [11,12]. This inherently makes a justice system for recipient prioritization based on these scores flawed. It has already been shown that different assays for serum creatinine result in different absolute values and differences in MELD scores which would be clinically important [11]. A similar situation has been shown for INR [12]. There is a movement in the UK now to change to one organ donor list for the whole of the UK and Ireland with a severity score similar to MELD (UKELD) as the determinant of priority. As outlined by Neuberger et al., the UK Liver Advisory Group has developed guidelines for selection criteria for liver transplantation and aims to introduce UKELD as a means of prioritizing patients for transplant [13]. The purpose of this study was to evaluate the degree of agreement of serum creatinine measurements and MELD scores derived from these values in liver transplant centres in the UK, as creatinine is a surrogate of renal function and and a key prognostic indicator. Materials and methods Study design The study was approved by the UK Liver Advisory Group, i.e. by all centres as an appropriate audit, given the preparation for UKELD and the concerns from all units. We invited all eight liver transplant units in the UK and Ireland to participate in the study, four of which accepted. The participating units were labelled A D, with only the study co-coordinator knowing the code. Selection was based on being a potential or current transplant candidate with cirrhosis. Each centre was asked to provide a blood sample from up to 10 patients who were either on the transplant list or who were being assessed for placement on the list. Each centre collected blood into 2 10ml lithium heparin or serum in serum separator (SST) bottles from each patient. This sample was then divided into aliquots of 2 ml each for transport to each of the other centres involved. Two 2-ml aliquots from each patient were sent to each participating centre. In each centre, the samples were re-run for measurement of creatinine and bilirubin. Thus, each patient had their creatinine and bilirubin measured four times. Centre C used the Dade Dimension technique for creatinine measurement, whilst the other centres used techniques based on the kinetic Jaffe technique (Table 1). The biochemistry department of all four centres participate in external assurance schemes UK National External Quality Assessment Service (UKNEQAS) and the Randox International Quality Assessment Scheme (RIQAS) which assess the stability of their calibration and within-method and all-method performance (accuracy). Each centre also runs regular internal quality control checks to ensure assay stability. The four centres had a mean inter-run C.V. (coefficient variation) of 1.8% and a mean intra-run coefficient variation of 1.2%. Most of the patients had a coagulation screen done at the same time; for those who did not, we used an INR value within 3months of the study date. The INR for each patient was only measured in their own hospital and not re-assessed in each of the centres. We used that same INR value to calculate the MELD score for that patient in each of the centres, thereby avoiding any potential impact that varying INR values may have on MELD score. In total, there were 36 patients evaluated between the four centres; three patients did not have an INR measured recently enough to include in the study. The results from all the centres were then collected. Creatinine and bilirubin were measured in micromoles per litre but converted into milligrams per decilitre for calculation of MELD scores from the official UNOS website ( In order to eliminate the possibility of variation in bilirubin measurement between centres affecting MELD scores, we calculated each patient's MELD score twice, once using the bilirubin from each centre and then using only the bilirubin value for each sample as measured at the Royal Free Hospital for calculation of MELD score. There was no significant variation between these two sets of MELD scores. Statistical analysis All data were analysed using the statistical package SPSS (version 10.0). Significance testing was two-sided and set to less than Wilcoxon signed-rank test was used for non-parametric evaluation between paired creatinine values and paired MELD scores. The degree of agreement between the different labs for creatinine was evaluated using the Bland and Altman method, i.e. by plotting the difference between the two scores against their mean [14]. Bland Altman plots give a visual representation of the degree of agreement and are the recognized way to assess this issue particularly in this context in which exact numerical equivalence would be expected. A 10% disagreement is arbitrary but is a useful rule of thumb and is recommended by Bland Altman [14]. We evaluated the absolute number and percentage of values outside 1.96 SD, and if the latter was 10%, this was considered very poor agreement. Results Patient characteristics The 36 patients had a median age of 54 (26 70) years, and 17 (47%) were men; the aetiology of liver disease included: alcohol (55%), primary biliary cirrhosis (20%), hepa-

3 962 C. Goulding et al. Table 2. Results for those with bilirubin 100μmol/L Centre A Centre B Centre C Centre D Sample INR Creatinine μmol Bilirubin μmol MELD Creatinine μmol Bilirubin μmol MELD Creatinine μmol Bilirubin μmol MELD Creatinine μmol Bilirubin μmol MELD Gender A F B F A M A F A F B F A F B M A F C F A F C F A M C F E M C M E M B M C M B M E F E F A M B M C M B M

4 Creatinine measurement and MELD scoring in liver transplant units 963 Table 3. Results for those with bilirubin >100μmol/L Centre A Centre B Centre C Centre D Gender Creatinine μmol Bilirubin μmol MELD Creatinine μmol Bilirubin μmol MELD Creatinine μmol Bilirubin μmol MELD Creatinine μmol Bilirubin μmol MELD Sample INR E F A M E F B M E F B M E F C F E F E M titis C virus (15%) and auto-immune hepatitis (10%). Patient results from all four centres are shown in Tables 2 and 3, divided into those with bilirubin 100μmol/L and those with bilirubin >100μmol/L. Laboratory measurements The same blood samples were assayed again in the four different centres. The median values (range) of creatinine, bilirubin and INR amongst the four labs A, B, C and D were as follows: (i) for creatinine (μmol/l): 100 (55 286), 99 (48 290), 92 (41 305) and 95 (36 297), respectively, (ii) for bilirubin (μmol/l): 41.5 (7 715), 44 (7 725), 40.5 (5 691) and47(5 799) and (iii) for INR: 1.3 [1 3] (Table 4). All comparisons of the differences of creatinine between centres A, B, C and D were statistically significant (Wilcoxon test: P < 0.05). The median MELD scores for centres A, B, C and D were 16, 17, 16 and 16, respectively. We then looked at median creatinine values in two groups: those with low bilirubin levels, i.e. 100μmol/L (n = 26), and those with high bilirubin values, i.e. > 100μmol/L. The median values (range) of creatinine for centres A, B, C and D in those with low bilirubin was 92 (59 286), 88 (56 290), 85 (48 305) and 85 (43 297), whilst in those with high bilirubin it was 117 (54 150), 115 (48 150), 97 (41 143) and 103 (36 142), respectively, Table 4. Thus, as expected, there was more variability in creatinine values with higher bilirubin values. Creatinine of centre A was significantly different compared to creatinine of centres B, C and D in 33 (92%), 34 (94.5%) and 35 (97%) samples, respectively (P < 0.01). Creatinine of centre B was significantly different compared to creatinine of centres C and D in 36 (100%) and 34 (94.5%) samples, respectively (P < 0.01). Creatinine of centre C was significantly different compared to creatinine of centre D in 35 (97%) samples (P = 0.019). These differences remained significant in both groups of patients separated according to bilirubin concentrations ( 100 or >100μmol/L). Agreement between creatinine values between the four centres The Bland and Altman method for degree of agreement for all creatinine values between the four centres showed less than 10% of values to be outside 2 SD. The agreement was poorer for the creatinine between centres A and D (8.3% outside 2 SD) and between B and D (8.3% outside 2 SD) and better for all the other combinations (0 5.5% outside 2 SD). The findings were similar when samples with bilirubin 100mol/L or >100μmol/L were analysed separately; however, agreement was very poor (>10% outside 2 SD) between centres B and D in the group of high serum bilirubin (>100μmol/L) (Figure 1). MELD scores calculated in the different labs As mentioned, the creatinine and bilirubin levels were converted to milligrams per decilitre, and then the website ( was used to calculate MELD scores (the INR was not re-checked in all four labs). The mean

5 964 C. Goulding et al. Table 4. Summary of results from all four centres Median (range) Centre A Centre B Centre C Centre D Creatinine (μmol/l) 100 (55 286) 99 (48 290) 92 (41 305) 95 (36 297) Bilirubin (μmol/l) 41 (7 715) 44 (7 725) 40 (5 691) 47 (5 799) Creatinine (bilirubin <100 μmol/l) 92 (59 286) 88 (56 290) 85 (48 305) 85 (43 297) Creatinine (bilirubin >100 μmol/l) 117 (54 150) 115 (48 150) 97 (41 143) 103 (36 142) INR MELD MELD scores from the various labs were as follows: A = ± 8, B = ± 8, C = 15.9 ± 8 and D = 16.6 ± 8. The MELD scores were significantly different between centre C and all other centres (P < 0.05). Again, we subdivided our results into two groups for further analysis: those with low bilirubin 100μmol/L and those with high bilirubin >100μmol/L. The mean MELD scores for centres A, B, C and D with low bilirubin values concentrations were: 12.4 ± 5.5, 12.3 ± 5.5, 12.2 ± 6 and 12.5 ± 5.7, respectively (P < 0.05 for MELD from centre D vs. centres B and C) and for those with high bilirubin values were: 25.3 ± 4.1, 25.3 ± 4.3, 24.5 ± 4 and 24.7 ± 3.6, respectively (P < 0.05 for MELD from centre Avs. centres C and D as well as for MELD from centre B vs. centres C and D). The difference between MELD scores was never greater than two MELD points. MELD A scores, compared to MELD B, MELD C and MELD D scores for example had two points of difference in none (0%), four (11%) and one (2.7%) case, respectively. The proportion of MELD A scores, which were different compared to MELD B, MELD C and MELD D scores, progressively increased with higher bilirubin values: for bilirubin 100μmol/L (n = 23), only one (4%) had a difference of two points; for bilirubin 100μmol/L (n = 10), three (33%) had a difference of two points. This trend was similar also for the other labs B, C and D when each was considered, in turn, as the reference score. The MELD score for each centre according to creatinine values above and below 100μmol/L was also assessed. When the creatinine value was <100μmol/L (n = 16), there was no statistically significant difference in MELD scores between any of the centres, except centres C and D (P < 0.008). However, when only those with creatinine values >100μmol/L (n = 17) were assessed, the MELD scores from centre C were significantly different from centre A (P = 0.04), centre B (P = 0.009) and centre D (P = ), reinforcing the point that discrepancies in biochemical measures and therefore MELD scores become more marked in those with the most abnormal values, i.e. the sickest patients. Discussion When the MELD scoring system was introduced, one of its strengths was proposed to be the fact that it comprised only three laboratory tests, which were simply and universally measured and easily reproducible. This was felt to assure that the same MELD score would be obtained no matter where the sample was analysed. MELD also had the advantage of taking renal function into account, as this is well known to be an important prognostic factor in liver disease [15]. In the MELD system, creatinine is used as a surrogate marker for renal function. Our study has shown significant variation in measurement of creatinine values and therefore MELD scores from the same blood samples re-tested in four different laboratories. The results of this study are clear-cut, and although it can be considered small, its findings are supported by a larger study by Lisman et al. using seven centres throughout Europe which had similar results, with variation in creatinine values between laboratories giving a difference of up to three MELD points in the same patient [16]. Cholongitas et al. also showed significant variation in creatinine values when different laboratory methods for creatinine measurement were used in the same centre [11]. Similar to the data in this paper, the difference became more marked as serum bilirubin concentrations increased, as bilirubin is a known chromogen that interferes with creatinine measurement [17]. This means that the error in measurement is worse the more severe the jaundice, i.e. in the sickest patients, with the highest MELD scores and therefore those most in need of a liver transplant. These discrepancies are sufficient to allow a situation in which a patient may die whilst on the waiting list, who may otherwise have received a transplant, if their blood was analysed in a different laboratory. The number of patients in our study with high serum bilirubin levels was small, perhaps explaining why the differences in MELD scores are not as marked as in the study by Cholongitas et al. As mentioned, even in laboratories using the same method of measurement, clinically significant differences in creatinine values can still occur due to variations in normal ranges [11]. Added to potential laboratory discrepancies is the issue that factors such as gender, muscle mass, age and race can also affect creatinine levels [18]. Creatinine is not an accurate predictor of renal function or glomerular filtration rate (GFR) [18,19]. A study from the Royal Free Unit has shown creatinine to be significantly underestimated in women compared to men correcting for a similar estimated GFR. Estimated GFR was compared to true GFR using EDTA Cr clearance. Thus, females with liver disease had lower GFR than males for the same creatinine value [20]. This suggests that females suffer a systematic discrimination when creatinine is to be used in scoring systems such as MELD. This has also been documented in the USA since the introduction of the MELD system, where women have a 30% increased in odds of dying or becoming too sick whilst on the transplant waiting list compared with men [21].

6 Creatinine measurement and MELD scoring in liver transplant units 965 Fig. 1. Bland Altman plots of the degree of agreement of creatinine measurements between all centres. The difference between the creatinine values is on the Y-axis; the average is on the X-axis. Although this current study did not assess variations in INR measurements in different units and the potential impact on resultant MELD scores, this has been an issue highlighted by Trotter et al., who demonstrated considerable variation in INR, and therefore in MELD score across laboratories, with variations of up to seven points in MELD

7 966 C. Goulding et al. score [12,22]. This is due to the fact that the thromboplastin substrate is not standardized for liver disease [23]. As far back as 2000, the American Association for the Study of Liver Diseases (AASLD) stated that additional research into standardization of prothrombin time (PT) for patients with liver disease was needed, and this is ongoing. Studies have shown that the variation in INR can be significantly reduced if the thromplastin used is calibrated against an INR using patients with liver failure (INR LD ) instead of those on oral anticoagulants [23,24]. This is one potential way of decreasing variation in MELD scores, although it could be technically cumbersome. Another suggestion has been to remove INR from MELD scoring as Heuman et al. [25] have shown that it does not significantly decrease the accuracy of MELD; however, given that we and other groups have now documented similar problems with creatinine, that is unlikely to be the solution [11,16,20]. Our study shows significant variation in creatinine values and therefore resultant MELD scores across laboratories in four UK liver transplant units, even between centres using the same laboratory method, which was the case for three of the four laboratories which used a modified Jaffe method. When one considers that MELD at very best is only correct approximately 80% of the time in estimating 3-month survival whilst on the liver transplant waiting list and add to this potential variations in all three components of the MELD score, i.e. creatinine, bilirubin and INR, due to different laboratory methods and variations due to gender, race and muscle mass, it becomes clear that the MELD system as it currently stands has problems. It is not robust enough to be an equitable system fulfilling the criteria of justice and transparency. This also would be the case for other current and future scores that use or will use these same variables, such as UKELD, if no standardization of measurement takes place [13]. If MELD or similar scores were to be introduced to the UK and Ireland for allocation of a shared organ pool, then standardization of laboratory techniques and normal values would have to be undertaken for all of the transplant units. Secondly, the use of alternative measurements of renal function such as cystatin-c in the formula or the use of measured GFR and corrected creatinine or the use of INR liver [23,24] would have to be considered to eliminate any gender, race or laboratory bias. An alternative solution would be to have all the blood samples analysed in a single laboratory, but this is likely to be impractical, especially in relation to INR measurement. Conflict of interest statement. None declared. References 1. Brown RS, Lake JR. The survival impact of liver transplant in the MELD era, and the future for organ allocation and distribution. Am J Transplant 2005; 5: Wiesner R, Edwards E, Freeman R et al. Model for end-stage liver disease (MELD) and allocation of donor livers. Gastroenterology 2003; 124: Kamath PS, Wiesner RH, Malinchoc M et al. A model to predict survival in patients with end-stage liver disease. Hepatology 2001; 33: Cholongitas E, Marrelli L, Shusang V et al. A systematic review of the performance of the model for end stage liver disease in the setting of liver transplantation. Liver Transplant 2006; 12: Freeman RB. MELD/PELD: one year later. Transplant Proc 2003; 35: Freeman RB, Wiesner RH, Edwards E et al. UNOS/OPTN Liver and Intestine Transplantation Committee. Liver Transplant 2004; 10: Merion RM, Schaubel DE, Dykstra DM et al. The survival benefit of liver transplantation. Am J Transplant 2005; 5: Ravaioli M, Grazi GL, Ballardini G et al. Liver transplantation with the MELD system; a prospective study from a single European centre. Am J Transplant 2006; 6: De La Mata M, Cuende N, Huet J et al. Model for end stage liver disease score based allocation of donors for liver transplantation; a Spanish multicentre experience. Transplantation 2006; 82: Ravaioli M, Masetti M, Ridolfi L et al. Laboratory test variability and model for end stage liver disease score calculation: Effect on liver allocation and proposal for adjustment. Transplantation 2007; 87: Cholongitas E, Marelli L, Kerry A et al. Different methods of creatinine measurement significantly affect MELD scores. Liver Transplant 2007; 13: Trotter JF, Brimhall B, Arjal R et al. Specific laboratory methodologies achieve higher Model for End Stage Liver Disease (MELD) scores for patients listed for liver transplant. Liver Transplant 2004; 10: Neuberger J, Gimson A, Davies M et al. Selection of patients for liver transplantation and allocation of donated livers in the UK. Gut 2008; 57: Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: Fraley DS, Burr R, Bernardini J et al. Impact of acute renal failure on mortality in end-stage liver disease with or without transplantation. Kidney Int 1998; 54: Lisman T, Van Leeuwen Y, Adelmeijer J et al. Interlaboratory variability in assessment of the model of end-stage liver disease score. Liver Int 2008; Cholongitas E, Shusang V, Marelli L et al. Renal function assessment in cirrhosis difficulties and alternative measurements. Aliment Pharmacol Ther 2007; 26: Levey AS, Perrone RD, Madias NE. Serum creatinine and renal function. Annu Rev Med 1988; 39: Perrone RD, Madias NE, Levey AS. Serum creatinine as an index of renal function: new insights into old concepts. Clin Chem 1992; 38: Cholongitas E, Marelli L, Kerry A et al. Female liver transplant recipients with the same GFR as male recipients have lower MELD scores a systematic bias. Am J Transplant 2007; 7: Moylan CA, Brady CW, Johnson JL et al. Disparities in liver transplantation before and after the introduction of the MELD score. JAMA 2008; 300: Trotter JF, Olson J, Lefkowitz J et al. Change of international normalised ratio (INR) and Model for End Stage Liver Disease (MELD) based on selection of clinical laboratories. Am J Transplant 2007; 7: Tripodi A, Chantaranqkul V, Primiquani M et al. The international normalized ratio for cirrhosis (INR (liver)) normalizes prothrombin time results for model of end stage liver disease calculation. Hepatology 2007; 46: Bellest L, Eschwege V, Poupon R et al. A modified international normalized ration as an effective way of prothrombin standardization in hepatology. Hepatology 2007; 46: Heuman DM, Minas AA, Habib A et al. MELD-XI: a rational approach to sickest first liver transplantation in cirrhotic patients requiring anti-coagulation therapy. Liver Transplant 2007; 13: 2 4 Received for publication: ; Accepted in revised form: