Mortality predicted by APACHE I1

Anaesthesia, 1996, Volume 5 1, pages 7 19-723 Mortality predicted by APACHE I1 The effect of changes in physiological values and post-icu hospital mortality D. R. GOLDHILL AND P. S. WITHINGTON Summary The contribution of physiological values to the APACHE II score was determined by retrospective analysis of I I 348 patients undergoing intensive care. Eleven physiological variables contributed a mean of 8.9 points, 54% of the total APACHE II score. The mortality ratio (observed hospital mortality/hospital mortality predicted by APACHE II) was 1.13. We altered the APACHE II scores and post-intensive care hospital mortality in order to examine the effect on the mortality ratio of these changes. Increasing scores by two or four points decreased mortality ratios to 1.OO and 0.89 respectively: decreasing scores by two or four points to a minimum of zero increased mortality ratios to 1.27 and 1.44 respectively. A 25% increase or decrease in post-intensive care hospital mortality changed mortality ratios to I.21 and 1.05 respectively. Physiological values vary with the timing of collection and accuracy of recording. Small consistent diflerences in scores cause potentially important changes in the mortality ratio. Unless data collection and the effect of management before and after intensive care are standardised, using mortality ratios to compare intensive care units is likely to be inaccurate and misleading. Key words Intensive care; scoring system, APACHE 11. Complications; mortality. Patients admitted to the intensive care unit (ICU) have a wide range of underlying pathologies and physiological abnormalities. Scoring systems have been developed in order to allow comparisons in outcome between these patients [I, 21. The most commonly used scoring system is APACHE I1 which assumes that there is a strong and consistent underlying relationship between acute physiological derangement and the risk of death during acute illness [3]. The APACHE I1 score is derived from 11 physiological variables (Table l), the Glasgow coma score (GCS) and the patient s age and chronic health status. The physiological variables score zero if normal and up to four points for abnormalities apart from the creatinine which scores double points (up to eight) in acute renal failure. Of the maximum 71 points, six depend on the patient s age, five on previous chronic health, 12 on the Glasgow Coma score and 48 on abnormalities in the physiological variables. The age and chronic health components of the APACHE I1 score are fixed but most of the physiological variables are goals of treatment. The worst physiological values within 24 h of ICU admission are scored and, with the possible exception of the white blood cell count and creatinine, it is often possible to improve values of the physiological variables with resuscitation. The patient s predicted mortality is calculated from their APACHE I1 score, a coefficient based on the reason for ICU admission (diagnostic category) and an additional weighting for emergency surgery. In his original paper on APACHE I1 [3] Knaus et al. stated: This scoring system can be used to... compare the efficacy of intensive care in different hospitals or over time. Recent analysis of a large ICU database suggests that there has been no noticeable improvement in the outcome of intensive care patients when APACHE I1 is used to predict mortality [4]. It is possible that there has been little or no benefit from the many developments in the management of D.R. Goldhill, MA, MB, BS, FRCA, Senior Lecturer, P. Stuart Withington, MB, BS, FRCA, MBES, Senior Lecturer, The Anaesthetics Unit, They Royal London Hospital, Whitechapel, London El 1 BB. Accepted 21 December 1995. 0003-2409/96/080719 + 05 $18.00/0 @ 1996 The Association of Anaesthetists of Gt Britain and Ireland 719

720 D.R. Goldhill and P.S. Withington critically ill patients. An alternative explanation is the failure of physiologically based intensive care scoring systems, such as APACHE 11, to account for the effects of treatment outside of the ICU. Assuming the values of the physiological variables do not deteriorate after admission to the ICU, a patient resuscitated with improved physiological values before ICU admission will have a lower APACHE I1 score on admission and therefore a lower predicted hospital mortality, than if the same patient is admitted to the ICU before resuscitation. If observed hospital mortality is unaffected by whether resuscitation takes place before or after ICU admission, the patient resuscitated before ICU will have a higher mortality ratio (observed hospital mortality/predicted hospital mortality) than the patient admitted to the ICU for resuscitation. In addition, as APACHE I1 is based upon hospital, not ICU, mortality, differences in outcome after ICU discharge may also affect the mortality ratio. In this way alterations in ICU care may be masked by changes in pre-icu resuscitation or post-icu management. We examined this hypothesis by analysing a large intensive care database. The contribution of the physiological variables was identified to determine whether pre-icu treatment had the potential to alter APACHE I1 scores. We then calculated the effect upon the predicted hospital.outcome and mortality ratio of changes in the values of physiological variables. The effect of altered post-icu mortality was also calculated by assuming a higher and a lower mortality after ICU discharge. Methods In the North Thames (East) region of the United Kingdom 19 lcus contribute information, including APACHE I1 scores, to a database. Standard agreed definitions are used when recording the information. The data are collected on a typeset form by marking boxes in the manner of a multiple choice paper. The worst values of inspired oxygen concentration, Pao, and Pacoz are recorded and the number of points for oxygenation calculated. All other physiological points are recorded by marking boxes indicating the range which encompasses the worst value of the variable. In each ICU an identified nurse or doctor has the responsibility for collecting the data. Training and support is provided by a full-time regional audit co-ordinator. Forms are sent to a central office where they are read by an optical mark reader (OMR) and the data entered into a computer database. Validation of the data Table 1. The 1 I physiological variables contributing to the APACHE I1 score. Maximum points Temperature 4 Mean arterial blood pressure 4 Heart rate 4 Respiratory rate 4 Oxygenation (either A-aDoZ or Pao?) 4 Arterial ph 4 Serum sodium 4 Serum potassium 4 Serum creatinine (with acute renal failure) 8 Haematocrit or haemoglobin 4 White blood cell count 4 begins with a visual check of the form at the base ICU and again at the central office. Simple error checking is incorporated into the OMR so that incomplete forms or those with mutually exclusive marks are rejected, and returned for completion. Further error checking takes place within the database to reject obvious errors such as patients discharged before admission, or duplication of records. Further checks are run within the database to identify anomalous data. Data entered into the database on all patients admitted to an ICU between January I 1992 and 31 July 1995 were analysed in an approved audit review. Data were excluded from four units that had entered fewer than 300 patients into the database, as it was felt that their patients may be unrepresentative and data less accurate due to unfamiliarity with the data collection method. In accordance with the criteria of Knaus et d. [3], patients admitted after cardiac surgery, with burns or under 16 years of age were not studied, nor were patients with a previous ICU admission within 6 months or where ICU or hospital outcome was unknown. If physiological data was unobtainable (e.g. ph when blood gases were not available) values were assumed to be normal. The forms were only read into the computer and incorporated into the database when information on all physiological values was complete. The reliability of the data recorded was assessed by reviewing the charts on a random sample of 100 patients from the Royal London hospital. APACHE I1 scores were calculated from the worst values within 24 h of ICU admission. The calculation of predicted mortality used the original equation and coefficients published by Knaus et a/. [3]. We calculated the number of points contributed to the APACHE I1 score by each of the variables. We further divided the data into 10 bands based on predicted mortality and measured the contribution made by the variables in each band. The mortality ratio (observed hospital mortality divided by predicted hospital mortality) was then plotted for each predicted risk of death band. The data were then reanalysed assuming that the sum of the patient s 11 physiological scores (Table 1) was increased by two or four, or decreased by two or four to a minimum of zero. The data were then grouped by predicted mortality. The information was further analysed assuming that there were either 25% greater or 25% fewer hospital deaths for patients discharged alive from the ICU. The mean and 95% confidence intervals for the mortality ratios were calculated as the observed hospital deaths divided by the mean and mean 95% confidence intervals of the predicted hospital mortality. Differences between observed and predicted numbers of hospital deaths were analysed using the Chi-squared test. Results The 19 lcus contributed a total of 21 152 complete ICU admission records. After excluding patients where hospital outcome was unknown (330), with previous ICU admission (2013), under 16 years of age (666), after cardiac surgery or with burns (6311) and from four ICUs contributing less than 300 patients (484), data on 11 348 patients remained for analysis with a median of 623 (range 319-1921) admissions per ICU. Review of 100 random charts demonstrated that there was a 20% likelihood of an inaccuracy in one of the I1 physiological variables. The Anaeslhesio. Volume 51. August 1996

~ ~~ ~~ Mortality predicted by APACHE II 121 Table 2. The percentage of patients with abnormalities (>one point) in the physiological variables for all patients and also grouped by predicted mortality. The mean score contributed by the variables is also shown and also that contributed by six variables judged to be most easily influenced by treatment (heart rate. mean arterial BP, resoiratorv rate. oh. oxvgenation and haemoelobinl All Predicted mortality (YO) patients 0-10- 20-30- 4& 50-60- 70-80- 90- Variahle Temperature Mean arterial BP Heart rate Respiratory rate Oxygenation PH Sodium Potassium Creatinine Haemoglobin White cell count Number of patients Mean score Mean score for six variables 39.5 48.5 54.4 29.2 43.3 10.6 25.1 29.3 31.3 27.7 45.0 1 I348 8.9 6.1 24.5 25.7 35.4 15.3 18.4 3.2 11.4 9.5 17.0 13.6 21.0 4486 3.8 2.8 40.2 46.4 53.8 27.4 38.0 8.3 22.7 24.6 33.0 23.5 43.2 1710 7.0 4.9 42.5 48.8 49.4 50.6 56.9 54.1 60.3 70.5 49.6 58.9 64.0 68.3 78.4 79.4 87.6 95.9 58.9 66.3 66.4 69.5 73.0 79.6 84.1 89.5 34.9 37.7 39.7 37.4 40.2 41.4 51.9 63.9 46.3 57.9 59.3 70.1 70.7 76.5 87.4 95.1 12.9 13.9 16.0 19.8 18.6 20.2 20.9 28.6 28.0 34.9 33.5 37.2 40.2 45.1 44.6 53.4 31.9 36.0 41.8 48.6 49.0 55.7 63.5 85.9 37.9 41.4 42.0 41.1 43.9 43.5 45.1 60.3 32.4 37.3 39.2 40.7 43.9 49.5 46.2 56.6 53.6 58.8 63.9 64.6 69.9 75.1 79.6 85.9 1114 777 648 545 522 481 597 468 9.3 11.0 12.3 13.2 14.3 15.8 18.3 24.1 6.4 7.6 8.3 8.9 9.6 10.6 12.4 15.4 most common error was for a variable to be incorrectly scored as normal (i.e. zero points). The percentage of patients with abnormalities (>one point) in the 11 physiological variables is shown in Table 2. The mean score contributed by the variables is also shown as is the mean score contributed by the six variables judged to be most easily influenced by treatment, (heart rate, mean arterial blood pressure, respiratory rate, ph, oxygenation, and haemoglobin). The 11 physiological variables contributed a mean of 8.9 points to the APACHE I1 score, 54% of the total. The six selected variables contributed a mean of 6.1 points, 37% of the total. Table 3 shows mortality ratio (95% CI) for the actual data and for an increase in the physiological score of two or four points, or a decrease of two or four points to a minimum of zero. Of the total 3692 deaths (32.5% of patients), 1021 (27.7% of the deaths) occurred in hospital after discharge from the ICU. Table 3 also shows the mortality ratio assuming 25% greater or 25% fewer deaths in hospital for patients discharged alive from the ICU. Figure 1 illustrates mortality ratios by predicted mortality. stratification and comparison of heterogenous patients [l, 21. Data on APACHE I1 scoring applied to 8796 patients in British and Irish ICUs were published in 1993 [4]. The overall mortality ratio of 1.O2 (95% CI 0.98-1.06) suggests that there has been no significant improvement in patient outcome since Knaus et al. published their seminal paper in 1985 [3]. However, there may be limitations in the APACHE I1 methodology so that improvements in care are not identified. One limitation identified by Rowan et al. [4] is that case mix can significantly affect the mortality ratio [5]. Another limitation may be consistent differences in recording the data. In Knaus s original paper [3] data were missing on 13% of admissions. Although the definitions, training and technique for data collection are the same in the ICUs contributing to our database, differences in resources and enthusiasm are likely to result in a range of data accuracy across the units. The most common error in our ICU is for a variable to be incorrectly scored as normal (i.e. zero Discussion Physiological-based scoring systems are widely used in intensive care to provide an objective prediction of outcome for a group of patients thus allowing description, Table 3. Mortality ratios (MR) and 95% confidence intervals for actual data and for an increase in physiological scores of 2 or 4 points, a decrease in physiological points of 2 or 4 to a minimum of 0. and an increase or decrease in post-icu hospital mortality Of 25%. MR (95% CI) Physiological points minus 4 points 1.44 (1.41 to 1.47)** Physiological points minus 2 points 1.27 (1.25 to 1.30)** 25% more deaths 1.21 (1.19 to 1.23)** Actual data 1.13 (1.11 to 1.15)** 25% fewer deaths 1.05 (1.03 to 1.07)** Physiological points plus 2 points 1.00 (0.98 to 1.02) Physiological points plus 4 points 0.89 (0.88 to 0.91)** **p < 0.01. observed versus predicted. MR, observed number of deaths divided by predicted number. Physiological points, points contributed by physiological variables (Table I). tl I I I I I I I I II 0-10- 2CL 30-40- 50-6b 70-80- 90- Predicted mortality (8) Fig. 1. Mortality ratios (MR) grouped by predicted mortality. This is shown for the actual data, for the addition of 2 or 4 points to the physiological scores, for the subtraction of 2 or 4 points from the physiological scores to a minimum of zero, and for an increase or decrease in post-icu hospital mortality of 25%. minus 4, -*-; minus 2, ---*---; 25% more. -0-; actual. -a-; 25% fewer, ---0---; plus 2, ---+---: plus 4, -+-. Anaeslhesia. Volume 51, August 1996

122 D.R. Goldhill and P.S. Withington points). If this is a consistent error then the APACHE I1 score and predicted mortality will be lowered resulting in a higher mortality ratio. A further limitation may be the effect of treatment before admission to the ICU or after discharge. In our analysis we have attempted to show the impact of such treatment. Our analysis was performed on a large database consisting of real patient data. The importance of the analysis lies not in the accuracy, or otherwise, of our data but in the effect that small, clinically achievable changes to the APACHE I1 score have on mortality predicted from this database. The database has also been used to illustrate the changes that would reflect alterations in post-icu hospital mortality. We chose to analyse the effect of altering the patients APACHE I1 scores by two or four points with the proviso that the sum of the physiological variables could not be less than zero. This is a relatively large change in the APACHE I1 score for patients with a low predicted mortality and a small change for those with a high predicted mortality. Even so, of the patients with low predicted mortality (&<lo%), 35% scored points for heart rate, 25% for temperature and 26% for blood pressure (Table 2). The average score for the physiological variables was 3.8 for this low risk group of patients. Relatively minor and frequent physiological abnormalities make a significant contribution to the APACHE I1 score. For example if the mean arterial blood pressure falls below 70 mmhg, the heart rate below 70 beat.min- or the haemoglobin below log.dl-, at least two points are scored. The average number of points contributed by physiological variables and the high percentage of patients with abnormalities in these variables demonstrate that pre-icu resuscitation could feasibly decrease the APACHE I1 score by two or more points, even in patients with low predicted mortality. The mean mortality ratio for the patients in our units is 1.13 (Table 3) indicating a higher observed hospital mortality than predicted. Our analysis demonstrates the dangers of using this figure to conclude that our units are performing badly. Pre-ICU resuscitation for emergency admissions may improve physiological values and aggressive intra-operative management detect and correct physiological abnormalities thus lowering predicted mortality. Data from 3 11 patients admitted to our ICU after trauma and brought to the hospital by helicopter between January 1991 and July 1992 provide some support for this hypothesis. Between the initial assessment at the scene and arrival in the emergency room there were improvements to normal values in 73% with a low blood pressure, 96% with a low oxygen saturation, 80% with an abnormally high or low heart rate and 98% with abnormal respiration. Much depends on the time at which the worst ICU results are recorded. Knaus ef al. refer to the initial 24 h after ICU admission [3]. In our group of ICUs we include data from 1 h before ICU admission, although in practice pre-icu admission data may not be available or recorded. Even with this 1 h inclusion, many emergency admissions arrive on the ICU several hours after the initial resuscitation. In some hospitals patients may be admitted to the ICU for resuscitation, while in other hospitals resuscitation will take place elsewhere. If the resuscitation is identical in efficacy and time of initiation and only differs in location, APACHE I1 scores for identical patients with identical outcomes will differ widely between hospitals. Thus predicted mortality will be different while observed mortality remains unchanged. Post-ICU care may also alter the mortality ratio by influencing the observed hospital mortality. In this way, neglect after ICU discharge will increase the mortality ratio. Alternatively, the percentage of hospital deaths after ICU may be related to the type of work undertaken by the hospital. For example, patients having major palliative cancer surgery, haematological malignancies, AIDS or end-stage chronic respiratory failure will have a high hospital mortality even if the acute ICU care is successful. With 27.7% of deaths occurring after discharge from the ICUs in this study, these considerations may noticeably influence the mortality ratio. We used an increase or decrease of 25% in post-icu deaths to illustrate this point. The potential impact on mortality ratios of small changes in the APACHE I1 score is such that great caution must be adopted in comparing results between different ICUs or even within the same ICU over a period of time. Meticulously matched case control analysis may provide a method of confirming whether differences in the mortality ratio are a result of changes in practice outside or within the ICU. It is also essential to ensure that the raw data on which the score is based is accurate and consistent. It is surprising that for a system which has been so widely adopted, there is still confusion over basic definitions. APACHE I1 was not designed to predict mortality for individual patients and our analysis cautions against using it for this purpose. The problem of treatment-influenced physiological variables was acknowledged by Knaus et af. in their original paper [3] when they stated that early recording of the values of physiological variables would make the score more independent of treatment. Other authors have commented on this shortcoming of the APACHE I1 system [6-81. The APACHE I11 scoring system [9] has examined the impact of gathering data before ICU admission, although results suggest that this has little effect on the predicted outcome. Despite all these potential inaccuracies the APACHE I1 system has been used and continues to be advocated as a system to rank ICUs by patient outcome [10-14]. Our analysis indicates that changes in management outside the ICU, or inconsistencies in data collection or accuracy, may have an important effect on the mortality ratio. Changes in ICU care may therefore not be detected by APACHE 11. Unless account is taken of these factors, the use of mortality ratios to rank ICUs in league tables is likely to be inaccurate and misleading. Acknowledgments We are grateful to the following ICUs which have contributed information to the database and have allowed us to include the data in the analysis. Basildon, Chase Farm, Chelmsford, Colchester, Harold Wood, Homerton, King George s, The Middlesex, Newham, North Middlesex, Oldchurch, Harlow, The Royal Free, The Royal London, St Bartholomew s, University College, Whipps Cross, The Whittington, The London Chest. The audit system on which the information in this paper is based is supported by the North Thames (East) Health Authority. Anaesthesia, Volume 51, August 1996

Mortality predicted by APACHE II 723 References [I] SUTER P, ARMAGANIDIS A, BEAUFILS F, BONFILL X, BURCHARDI H, COOK D, FAGOT-LARGEAULT A, THIJS, L, VESCONI S, WILLIAMS A. Predicting outcome in ICU patients. Consensus conference organized by the ESICM and the SRLF. Intensive Care Medicine 1994; 20 390-7. [2] SENEFF M, KNAUS WA. Predicting patient outcome from intensive care: a guide to APACHE, MPM, SAPS, PRISM, and other prognostic scoring systems. Journal of Intensive Care Medicine 1990; 5 33-52. [3] KNAUS WA, DRAPER EA, WAGNER DP, ZIMMERMAN JE. APACHE 11: a severity of disease classification system. Critical Care Medicine 1995: 13: 818-29. [4] ROWAN KM, KERR JH, MAJOR E, MCPHERSON K, SHORT A, VESSEY MP. Intensive Care Society s APACHE I1 study in Britain and Ireland-11: Outcome comparisons of intensive care units after adjustment for case mix by the American APACHE I1 method. British Medical Journal 1993; 307: 977-8 1. [5] GOLDHILL DR, WITHINGTON PS. The ability of APACHE I1 to adjust for casemix difference. In press. Intensive Care Medicine 1996. [6] BOYD 0, GROUNDS RM. Physiological scoring systems and audit. The Lancet 1993; 341: 15734 [7] DRAGSTED L, JORGENSEN J, JENSE N-H, BONSING E, JACOFISEN E, KNAUS WA, QVIST J. Interhospital comparisons of patient outcome from intensive care: Importance of lead-time bias. Critical Care Medicine 1989; 17: 418-22. [8] ESCARCE JJ, KELLEY MA. Admission source to the medical intensive care unit predicts hospital death independent of APACHE I1 score. Journal of the American Medical Association 1990; 264: 2389-94. [9] KNAUS WA, WAGNER DP, DRAPER EA, ZIMMERMAN JE, BERGNER M, BASTOS PG, SIRIO CA, MURPHY DJ, LOTRING T, DAMIANO A, HARRELL FE. The APACHE I11 prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991; 100: 1619-36. [lo] KNAUS WA, DRAPER EA, WAGNER DP, ZIMMERMAN JE. An evaluation of outcome from intensive care in major medical centers. Annals of Internal Medicine 1986; 104: 410-18. [ll] KNAUS QA, WAGNER DP, ZIMMERMAN JE, DRAPER EA. Variations in mortality and length of stay in intensive care units. Annals of Internal Medicine 1993; 118: 753-61. [I21 CHANG RSW, BIHARI DJ. Physiological scoring systems and audit. Lancet 1993; 342 306. [ 131 PALAZZO M, SONI N, HINDS C. Physiological scoring systems and audit. Lancet 1993; 342 307. [ 141 HOLT AW, BERSTEN AD, WORTHLEY LI, VEDIC AE. Physiological scoring systems and audit. Lancer 1993; 342 307-8. Anaesthesia, Volume 5 I, August 1996