CLINICAL INVESTIGATIONS

Transcription

1 1040 Olsson and Lind d REMS AND APACHE II CLINICAL INVESTIGATIONS Comparison of the Rapid Emergency Medicine Score and APACHE II in Nonsurgical Emergency Department Patients Abstract Thomas Olsson, MD, Lars Lind, MD, PhD Objectives: To improve the Rapid Acute Physiology Score (RAPS) as a predictor of in-hospital mortality in the nonsurgical emergency department (ED) by including age and oxygen saturation, and to compare this new system, Rapid Emergency Medicine Score (REMS), with the Acute Physiology and Chronic Health Examination (APACHE II) with reference to predictive accuracy. Methods: This was a prospective cohort study. One hundred sixty-two critically ill patients consecutively admitted to the intensive care unit (ICU) during the period of one year, and 865 nonsurgical patients presenting to an adult emergency department (ED) and admitted to a medical department of a 1,200-bed university hospital during two months, were enrolled. For all entries to the ED, RAPS was calculated and developed to include noninvasive peripheral oxygen saturation and patient age (REMS), as well as laboratory tests (APACHE II). These scores were calculated for each patient. Results: REMS was found to be superior to RAPS in predicting inhospital mortality both in the critically ill patients admitted to the ICU and in the total sample (area under receiveroperating characteristic curve [AUC] for REMS compared with for RAPS, p \ 0.001). An increase of 1 point in the 26-point REMS scale was associated with an odds ratio of 1.40 for in-hospital death (95% confidence interval ¼ 1.36 to 1.45, p \ ). The more advanced APACHE II was not found to be superior to REMS (AUC: , p ¼ 0.218). Conclusions: RAPS could be improved as a predictor of in-hospital mortality in the nonsurgical ED by including oxygen saturation and patient age to the system. This new scoring system, REMS, had the same predictive accuracy as the well-established, but more complicated, APACHE II. Key words: scoring system; APACHE II; REMS; mortality prediction; emergency department; epidemiology; cohort. ACADEMIC EMER- GENCY MEDICINE 2003; 10: The emergency department (ED) is the main gateway to the medical system and, subsequently, to health care expenditures. The future will undoubtedly see ongoing analyses of the balance among access, quality, and cost for populations throughout the health care system. Providers will increasingly come to rely on an evidence-based approach to health care decisions, using data from population-based research to justify promising new therapies and abandon treatments that add little to patient outcome. The outcomes measured in emergency medicine research are often short-term (i.e., during the ED course), with less attention being paid to the impact that an ED visit has on an episode of illness that includes other sites of care or long-term follow-up. Although short-term outcomes such as admission rate, speedy changes in physiology as a result of ED From the Department of Internal Medicine, University of Uppsala, Uppsala, Sweden (TO); and The Research and Development Unit (Jamtland County Council), Östersund, Sweden (LL). Received April 2, 2003; revisions received April 21 and May 9, 2003; accepted May 12, Address for correspondence and reprints: Thomas Olsson, MD, Ostersund Hospital, Department of Internal Medicine, SE Ostersund, Sweden. thomas.olsson@jll.se. doi: /s (03) treatment, or time to treatment are important, population-based data will become more important in the future. Several scoring systems for the assessment of the severity of illness have been advanced during the past few decades. 1 Several authors have described the wide range of uses of predictive instruments. 2 9 Although scoring systems have been applied to trauma patients, no scoring system has been specifically developed for nonsurgical patients presenting to the ED. A severity of disease classification in the ED combined with an accurate description of the disease could prognostically stratify acutely ill patients and assist investigators comparing the success of new forms of therapy. This scoring index also could be used to evaluate the use of hospital resources and compare the efficacy of different EDs in a short- as well as a long-term perspective. The severity of illness classification system APACHE II (Acute Physiology and Chronic Health Evaluation), described by Knaus et al. in 1985; 14 uses a point score based on 12 routine physiologic measurements, together with age and previous health status, for use on intensive care patients. APACHE II has been validated in both general and surgical intensive care patients. However, the APACHE II

2 ACAD EMERG MED d October 2003, Vol. 10, No. 10 d score includes several blood chemistry variables and is therefore not suitable for quick scoring in the ED. The Rapid Acute Physiology Score (RAPS), an abbreviated version of APACHE II, including the physiologic variables pulse rate, blood pressure, respiratory rate, and Glasgow Coma Scale (GCS) score, has previously been evaluated as an out-ofhospital scoring system after being used on a group of helicopter-transported patients. 24 The most significant advantage of the RAPS system as a prognostic tool at the ED would be the simplicity of the scoring procedure, because the four parameters can easily be collected even in an emergency. It would, however, be of great value to be able to improve the predictive accuracy of RAPS without complicating the system or rendering it less accessible. Body temperature and peripheral oxygen saturation are easily obtained at the ED. Chronological age is a well-documented risk factor for death caused by acute illness, independent of the severity of disease. 14 Three main questions were raised before the start of this study: First, could the abbreviated severity of disease classification system RAPS, created for use in the out-of-hospital setting, be useful in the ED for predicting in-hospital mortality and the length of hospital stay (LOS) in nonsurgical patients? Second, is it possible to modify RAPS to provide a more potent scoring system (the Rapid Emergency Medicine Score [REMS]) by including age and one or two parameters easily obtained by modern technology (oxygenation and body temperature) for the purpose of predicting in-hospital mortality? Third, could REMS, with its simplicity and fewer variables, perform as well as APACHE II in the nonsurgical ED? To answer these questions, measurements were taken in more than 1,000 nonsurgical ED patients. TABLE 1. Subjects (n) and In-hospital Deaths (n) for Different Coexistent Diseases and Medications in the Nonsurgical Patients Attending the Emergency Department (ED) Comorbidity and Medication Usage Subjects (n) In-hospital Deaths (n) Heart failure DM (I and II) Rheumatologic disease 65 3 Alzheimer s disease Multiple sclerosis 6 1 Parkinson s disease 18 1 Hematologic malignancies 12 3 Metastatic cancer 26 4 Nonmetastatic cancer IBD 9 1 Cirrhosis 8 2 Dialysis 8 2 COPD 39 5 Steroid therapy 48 7 Insulin therapy Nitroglycerine therapy Anticoagulant therapy 74 5 Chemotherapy 14 4 Antiepileptic therapy 35 2 Neuroleptic therapy 65 8 DM ¼ diabetes mellitus; IBD ¼ irritable bowl disease; COPD ¼ chronic obstructive pulmonary disease. ordinary medical department (n ¼ 758), to a general ICU (n ¼ 9), to a coronary care unit (n ¼ 84), or to a neuro-icu (n ¼ 15) were considered during the period January 1, 1996, to March 1, 1996, consecutively. These two groups of patients then were merged to form the total sample to achieve a sufficient number of cases (n ¼ 116). Furthermore, this procedure provided us with the opportunity to compare REMS METHODS Study Design. This was a prospective, observational study of a consecutive cohort including nonsurgical ED patients presenting to a large urban ED and admitted to a medical department over a two-month period, and critically ill patients admitted directly to the intensive care unit (ICU) during the period of one year. The study was approved by the Institutional Review Board for Human Research. Study Setting and Population. The study was conducted in a 1,200-bed university hospital. The number of subjects and deaths, different reasons for admittance to the ED, medications, and coexistent diseases are all presented in Tables 1 and 2. Adult nonsurgical patients were recruited from two sources. First, 185 nonsurgical, critically ill patients referred to the ICU from November 1, 1995, to November 1, 1996, were included. Second, 885 adult patients at the nonsurgical ED who were admitted either to an TABLE 2. Subjects (n) and In-hospital Deaths (n) for Different Symptoms on Admittance to the Emergency Department (ED) Reasons for Attending the ED Subjects (n) Mortality (n) Chest pain Stroke 92 8 Coma Dyspnea General weakness Hyperglycemia 35 2 Asthma 5 1 Fever 14 0 Cough 17 5 Arrhythmia 24 0 Vertigo 56 1 Syncope 28 0 Epileptic seizures 16 0 Hypoglycemia 1 0 Nausea, diarrhea, pain in the leg, anemia Total 1,

3 1042 Olsson and Lind d REMS AND APACHE II TABLE 3. Baseline Characteristics and Hospital Course for the 1,027 Patients (Total Sample) Presenting to the Emergency Department (ED) Mean SD Min Max Median Age (years) Length of stay (days) Gender (female) 50.4% In-hospital mortality 11.3% Mortality within 48 hours 5.8% SD ¼ standard deviation. and APACHE II in critically ill patients as well as in patients admitted to a non-icu medical department. Study Protocol. A database was available from our ED with REMS scores on every patient admitted to the ED from November 1, 1995, to November 1, 1996 (n ¼ 11,751). However, the major aim of the present study required both REMS and APACHE II scores, and there were no resources allocated to determine both for such a large number of patients. We compromised by taking the REMS score prospectively, even in the 185 patients who were referred directly to the ICU during one year, and by collecting the laboratory tests of the APACHE II retrospectively in both groups. The 185 patients were involved in the following way: A call was made from the out-of-hospital setting (ambulance, helicopter) to the ED, after which the nurse in charge informed the ICU about the incoming patient (general ICU: n ¼ 77; neuro-icu: n ¼ 23; coronary care unit: n ¼ 62) and noted this in the logbook. This was our method of identifying the worst subjects. These patients were in very poor condition and were brought to the ED for evaluation, but were transported directly to the ICU. However, our nurse in charge had been instructed about the procedure of taking the physiologic parameters in REMS (respiratory rate, heart rate, blood pressure, peripheral oxygenation, and GCS score), so these parameters were taken in the ED en route to the ICU. If the reason for admission was cardiac arrest (n ¼ 9) and the patients could not be resuscitated in the field, they were excluded, because these patients, by definition, would achieve a maximal score and thereby would skew the analyses. In the first group, 20 of the 885 patients had more than one physiologic parameter missing and were excluded. If only one parameter was missing (n ¼ 62), the data were collected at a later date from their medical records. If this single parameter was not mentioned in the record (n ¼ 8), it was regarded as normal (respiratory rate, n ¼ 4; pulse rate, n ¼ 4). The mortality rate (6SD) in this subgroup (n ¼ 865) was 7.5% (60.26), and 4.2% (60.2) died during the first 48 hours (n ¼ 62). In the second subgroup, 23 of the 185 patients had more than one REMS-variable missing and therefore were excluded. The mortality rate in this subgroup of the population (n ¼ 162) was 31.5% (60.47), and 14.8% (60.36) died during the first 48 hours (n ¼ 62). Thus, 43 patients were excluded, and the remaining total sample consisted of 1,027 patients. Measurements. APACHE II uses a point score based on values of 12 routine physiologic measurements, as well as age and previous health status. 14 The variables included in the APACHE II system are: body temperature, mean arterial pressure calculated from systolic and diastolic blood pressure, heart rate, oxygenation of arterial blood (PaO 2 ), arterial ph, serum sodium, serum potassium, serum creatinine, hematocrit, white blood cell count, and GCS score. The maximal APACHE II score is 71. RAPS was developed by taking the APACHE II elements easily obtained in the out-of-hospital settings. 24 These variables were pulse rate, blood pressure, respiratory rate (number of breaths during 30 seconds), and GCS score. The scoring procedure was identical to that of APACHE II except for GCS score, the relative value of which was reduced by two thirds compared with the APACHE II. The scoring range for each variable was 0 to 4, and the maximal score was 16 in the RAPS system. In our attempt to improve the predictive power of RAPS, peripheral oxygen saturation (0 4 points) and body temperature (0 4) were added to the four RAPS variables. Age also was added to RAPS, being TABLE 4. Univariate and Multiple Logistic Regression for All Parameters in RAPS and for Age, Body Temperature, and Oxygen Saturation Univariate Analysis Multivariate Analysis Variable Odds Ratio 95% CI p-value Odds Ratio 95% CI p-value 0 4 saturation , , respiratory frequency , , pulse frequency , , body temperature , , coma , , blood pressure , , age , , Odds ratios for in-hospital mortality are given for an increase of 1 point in the score. RAPS ¼ Rapid Acute Physiology Score; 95% CI ¼ 95% confidence inteval.

4 ACAD EMERG MED d October 2003, Vol. 10, No. 10 d TABLE 5. The Scoring Procedure for the Parameters Used in the Rapid Acute Physiology Score (RAPS): Peripheral Oxygen Saturation, Body Temperature, and Age High Abnormal Range Low Abnormal Range Physiologic Variable þ4 þ3 þ2 þ1 0 þ1 þ2 3þ þ4 Body temperature [ \30 Mean arterial pressure [ \49 Heart rate [ \39 Respiratory rate [ \5 Peripheral oxygen saturation \ [89 Glasgow Coma Scale score \ [13 Total sum of scoring points Assign points to age as follows: Age Points \ [74 6 weighted as in APACHE II (0 6 points). The nurse in charge of the patients applied the six-point standard examination within 20 minutes following ED arrival. The variables in APACHE II that were not collected in the REMS, (i.e., serum sodium, potassium, creatinine, hematocrit, and white blood cell count) were analyzed from blood samples drawn in the ED. Arterial ph was not used in the scoring system because this variable is not measured routinely in the ED. Data Analysis. A multivariate logistic regression analysis was performed to identify independent predictors of in-hospital mortality. REMS was defined based on that analysis (Table 2). The prediction of inhospital death by the RAPS and REMS systems was thereafter assessed by univariate logistic regression. The Spearman rank correlation method was used to determine the relationships between the scoring systems. Validity was assessed by using the so-called splitsample technique. 25 The total sample was split into two equal parts and evaluated independently. Discriminatory power refers to a model s ability to distinguish whether a patient died or survived. It is generally measured by the area under the receiver operating characteristic (ROC) curve, which plots the sensitivity against the specificity of the model over a wide range of mortality cut-off points. As the area under the curve (AUC) increases from 0.5 (random chance) to 1 (perfect discrimination), the accuracy and predictive power of the model improves. Generally, an AUC of 0.8 or better is expected for mortality predictions in current models, and scores of 0.7 or less are considered to be unacceptable. The standard error Figure 1. Receiver operating characteristic (ROC) curves graphically highlight the predictive power for each test. Sensitivity is plotted on the vertical axis, whereas 1ÿspecificity (equal to the false-positive rate) is plotted on the horizontal axis. The area under the curve determines the discriminatory power of the test. The ROC curve for the Rapid Emergency Medicine Score (REMS) systems ( d ) exceeded the recommended area for an effective test (area, ) and showed a superior discriminating power (p \ 0.001) compared with the Rapid Acute Physiology Score (RAPS) ( m ) ( ). 27 TABLE 6. Means with Standard Deviations for Age, Length of Stay (LOS) in Hospital, and Scoring Sums for Survivors and for Those Who Died during Their Hospital Stay Survived (n ¼ 1,027) Dead (n ¼ 116) p-value Age (years) \ LOS (days) \ REMS \ RAPS \ APACHE II \ REMS ¼ Rapid Emergency Medicine Score; RAPS ¼ Rapid Acute Physiology Score; APACHE II ¼ Acute Physiology and Chronic Health Evaluation.

5 1044 Olsson and Lind d REMS AND APACHE II TABLE 7. Odds Ratios for Each Rapid Emergency Medicine Score (REMS) Point Increase, Number of Subjects, and Number of Deaths for Different Age Groups Subgroups of the Population Odds Ratio 95% CI p-value Subjects (n) Mortality (n) Age \45 years Age years , Age years , Age years , Age [74 years , Total , , % CI ¼ 95% confidence interval. of the mean (SEM) and p-values for the ROC curves, as well as comparisons of them, were calculated by the Hanley and McNeil methods. 26,27 Calibration of the model is an evaluation of the extent to which the estimated probabilities of mortality of the model correspond to observed mortality rates. Calibration of REMS and APACHE II was evaluated with the Hosmer-Lemeshow goodness-of-fit test. 28,29 Statistic analyses were performed using the Stat- View 5.0 package for Windows (SAS Institute, Inc., Cary, NC) and the Statistical Package for the Social Sciences (SPSS) for Windows (SPSS Inc., Chicago, IL). p-values below 0.05 were considered significant. RESULTS All six physiologic measurements in the total sample were available in 95.2% of the cases. Basic characteristics of the 1,027 patients comprising the cohort are given in Table 3. Univariate logistic regression showed all six measured physiologic parameters and age to be significant predictors of mortality in the total sample. When multivariate analyses were performed, body temperature and blood pressure did not independently predict mortality, whereas the other four parameters did. Blood pressure was not removed from the scoring system because it is incorporated in both APACHE II and RAPS, and is by tradition routinely measured in all patients. Oxygen saturation and age were both strong predictors in the multiple logistic regression analysis (Table 4). The modified system (REMS) was therefore defined as the sum of coma, respiratory frequency, oxygen saturation, blood pressure, and pulse rate (maximal score being 4 for all) and age (maximal score being 6) (for details, see Table 5). Figure 2. Correlation between the two scoring systems Rapid Emergency Medicine Score (REMS) and Acute Physiology and Chronic Health Evaluation (APACHE II) (Spearman rank coefficient r s ¼ 0.87, p ¼ ). REMS was a powerful predictor of in-hospital mortality (n ¼ 116) in the sample, resulting in a likelihood ratio chi-square value of (p \ ) in the logistic regression model and an odds ratio (OR) of 1.58 for each point increase (95% confidence interval [CI] ¼ 1.48 to 1.70). RAPS was also a significant predictor of in-hospital death, with a likelihood ratio chi-square value of 273 (p \ ) and an OR of 1.77 (95% CI ¼ 1.62 to 1.93). The ROC curve for the REMS systems exceeded the recommended area for an effective test (area, ) and showed a superior discriminating power (p \ 0.001) compared with RAPS ( ) (Figure 1). Means with standard deviations (SDs) for scores, age, and LOS for those who survived and those who died during their hospital stay are shown in Table 6. REMS appeared to be a solid system throughout different age spans, grouped as in the APACHE age score. In the group with the youngest patients, there were only three deaths and, thus, REMS was not evaluated in that subgroup regarding in-hospital mortality (Table 7). There was no difference in ORs between men and women. REMS was further validated by the so-called splitsample method, i.e., the predictions made on the first 50% (n ¼ 514) of the patients included in the study were compared with the second half (n ¼ 513) of the population. The ORs and their 95% CIs for the two parts of the population were similar and are presented in Table 8. TABLE 8. Validation of the Rapid Emergency Medicine Score (REMS) with Split-sample Technique Odds Ratio 95% CI Likelihood Ratio p-value No. of Cases No. of Dead REMS in first 50% of the sample , \ REMS in the second half of the sample , \ Odds ratios for each point increase in both the sample sets. 95% CI ¼ 95% confidence interval.

6 ACAD EMERG MED d October 2003, Vol. 10, No. 10 d Figure 3. Receiver operating characteristic (ROC) curves for the Rapid Emergency Medicine Score (REMS) ( d ) ( ) and Acute Physiology and Chronic Health Evaluation (APACHE II) ( m ) ( ) regarding in-hospital mortality did not differ (p-value: 0.218). 27 APACHE II was also a significant predictor of inhospital death in the total sample, with a likelihood ratio chi-square value of (p \ ) and an OR of 1.25 for each point increase (95% CI ¼ 1.21 to 1.29). As shown in Figure 2, there was a strong correlation between REMS and APACHE II. Figure 3 shows the ROC curves for REMS and APACHE II regarding in-hospital mortality in the total sample. The two methods showed good discrimination, with no difference between the AUCs ( SEM for REMS and for APACHE II). When the two areas were compared by the Hanley-McNeil method, the p-value was One clear cut-off point could be defined in REMS as well in the APACHE II system (Figures 4 and 5). All patients presenting with REMS \6 and APACHE II \6 points survived. Model calibration, as indicated by the Hosmer- Lemeshow goodness-of-fit test, demonstrated that for REMS and APACHE II, there were no differences between observed and predicted mortality (chi-square value for REMS: 9.3, p ¼ 0.23; chi-square value for APACHE II: 8.7, p ¼ 0.36) (see Tables 9 and 10). When the two subgroups, the 162 critically ill patients directly admitted to the ICU, and the 865 nonsurgical patients admitted to an ordinary medical department were evaluated separately, discriminatory power (AUC) and ORs for APACHE II and REMS were similar to those presented for the total sample (Table 11). DISCUSSION Figure 4. Frequency distribution of the Rapid Emergency Medicine Score (REMS) (top panel ) and in-hospital mortality in the subsample. The present study showed that RAPS developed for the out-of-hospital settings was a predictor of inhospital death in nonsurgical patients admitted to the ED. However, when the RAPS sytem included information on peripheral oxygen saturation and age, as the REMS system, predictive ability improved considerably. Because peripheral oxygen saturation is measured in all patients in the ED, it would be very easy to enlarge the RAPS system by adding this parameter. Because age was a strong predictor of mortality in the multiple regression model, it seemed natural to incorporate that variable, valued as originally described in the APACHE II. Blood pressure did not independently predict mortality in the present study. In the original description of APACHE II, however, BP was measured invasively and, in most subjects, continuously. In this study, precision may have been lost by the calculation of mean blood pressure from a single noninvasive measurement of systolic and diastolic blood pressure. APACHE II is probably the most widely used and extensively evaluated scoring system for assessing the severity of illness, which has been developed for ICUs during the past three decades. In view of the vast experience connected with this scale in the ICU, we

7 1046 Olsson and Lind d REMS AND APACHE II TABLE 9. Calibration of the Rapid Emergency Medicine Score (REMS) with the Hosmer-Lemeshow Goodness-of-fit test 22 Comparing the Observed Mortality with the Expected Mortality within Deciles of Risk Deciles Alive Dead of Risk Observed Expected Observed Expected Total A 2 x 10 table is produced with ten strata of probabilities. The expected outcomes then were compared with the observed outcome for each decile. The calibration turned out to be acceptable (x 2 value: 9.3, p ¼ 0.23). Figure 5. Frequency distribution of the Acute Physiology and Chronic Health Evaluation (APACHE II) score (top panel ) and in-hospital mortality in the subsample. conducted a comparative study between APACHE II and REMS in the nonsurgical ED. Both REMS and APACHE II appeared to be significant predictors of in-hospital mortality in this study group. The OR was higher for REMS than for APACHE II, possibly because of the lower maximal score in REMS (26 compared with 40 for APACHE II). It is not possible to compare the predictive accuracy in terms of ORs when the scales are different. A strong point of this study was that all three major demands to be considered when developing a new severity of disease classification system were fulfilled. First, the validity was evaluated by the split-sample technique. 25 This method has reached standard level and, as such, is generally accepted by the medical community. Second, its discriminatory ability was assessed and presented in ROC curves, and these were compared using an established method. Third, the calibration of the scoring system, i.e., the evaluation of the extent to which the estimated probabilities of mortality of the model correspond to observed mortality rates, was calculated with the Hosmer- Lemeshow goodness-of-fit test. 28 Most of the widely used scoring systems created for the ICU setting are evaluated by this method, and both APACHE II and REMS appeared to have adequate calibration in this study (p[ 0.05). Thus, the established methodology in this field 30 was strictly followed in the process of developing REMS and comparing it with APACHE II. Disease-specific scoring systems already exist, such as those describing acute coronary syndromes, 31,32 stroke, 33 and asthma, 34 but most of these specific systems require data to be collected that are not easily available in the ED. The REMS system offers a speedy and easily performed scoring procedure with parameters accessible in the ED, and is also independent of future treatment. The APACHE II and other extensively studied scoring systems for the ICU setting are being used to stratify patients into equal groups TABLE 10. Calibration of the Acute Physiology and Chronic Health Evaluation (APACHE II) with the Hosmer-Lemeshow Goodness-of-fit Test 22 Comparing the Observed Mortality with the Expected Mortality within Deciles of Risk Deciles Alive Dead of Risk Observed Expected Observed Expected Total A 2 x 10 table was produced with ten strata of probabilities. The expected outcomes then were compared with the observed outcome for each decile. The calibration turned out to be acceptable (x 2 value: 8.7, p ¼ 0.36).

8 ACAD EMERG MED d October 2003, Vol. 10, No. 10 d TABLE 11. Odds Ratios (ORs) with 95% Confidence Intervals (95% CIs) for Each Point Increase in Acute Physiology and Chronic Health Evaluation (APACHE II) and Rapid Emergency Medicine Score (REMS) and Their Areas under the Receiver Operating Characteristic (ROC) Curves (AUCs) with Standard Errors of the Mean (SEMs) for Both the Subsamples 162 Critically Ill Patients 865 Nonsurgical Patients OR 95% CI p-value AUC SEM OR 95% CI p-value AUC SEM APACHE II ,1.27 \ ,1.31 \ REMS ,1.69 \ ,1.75 \ As in the total sample, there were no differences in discriminatory ability (AUC) between APACHE II and REMS in either of the two subpopulations (p [ 0.05) according to the method described by Hanley and McNeil. 27 for prospective trials, to compare performances and different treatment between units or hospitals, for quality-assurance purposes, and for resource utilization analyses. 2 9 REMS could probably be used for similar tasks in the ED. Fewer measurements are taken in REMS than in APACHE II, and REMS is less expensive and can be calculated quicker, as a result of the fact that no time-consuming blood chemistry analyses must be performed. Furthermore, the similarity of the discriminatory power and the calibration of the two systems is a strong argument for using the simpler scoring system, REMS, at the ED. LIMITATIONS These data represent patients from a nonsurgical ED at a university teaching hospital. We must therefore be cautious in generalizing the results presented here with a nonuniversity hospital or surgical ED. A subpopulation of critically ill patients (n ¼ 162) during a period of one year were added to the consecutively included 865 medical patients during a period of two months. The purpose was to achieve a sufficient number of cases. We compared the two scoring systems in the total sample as well as in the two subgroups separately, and found that the results regarding predictive accuracy were similar in the different samples. Therefore, we were able to conclude that the casemix compiled in the present prospective, consecutive study could well be justified. If a model consists entirely of patients at one particular hospital, as in our study, it holds a significant potential selection bias. The model will then incorporate any peculiarities of that specific group, undiluted by patients from other institutions. To rectify any such flaws, REMS must be tested in a multicenter study. The most controversial potential use for predictive models is to use them to affect individual case management, especially for issues of withholding or withdrawing treatment. 35,36 In the ED, however, a morally significant difference rightfully exists between the withholding and withdrawal of medical treatment. In the ICU setting, the withholding of further treatment is done quietly, often without input from the patient, whereas the withdrawal of ongoing medical treatment can be more obvious and difficult. 37 This situation is reversed in the ED, where there is often a lack of information concerning the identity, medical condition, and wishes of the patient, as well as special societal expectations about the nature of emergency care. However, the standard procedure in many hospitals to date is that the nurse in charge triages the patients regarding symptoms, e.g., patients with chest pain are prioritized before those who are dizzy. A combined system using both the symptoms and REMS could be of value in this work and provide both the nurse and the physician with a better and more reliable tool for triaging in the ED. Specific cutoff points in the score might define those who are critically ill and must be taken care of immediately. Turner et al. claimed that scoring patients later in their illness allows better predictability and decision making for individual patients. 37 To achieve maximum benefit from severity-of-illness scores, they should be repeated on successive days following admission. REMS is particularly well suited for this purpose with its simplicity and quickness, and has the potential of becoming a system for following the patient s status through continuous monitoring during transport, in the ED, and later during the patient s hospital stay. CONCLUSIONS The present study revealed REMS as being a significant predictor of in-hospital mortality in nonsurgical patients in the ED. REMS was superior to RAPS regarding explanatory power. Both APACHE II and REMS showed adequate and similar predictive accuracy, but REMS is suggested in preference as a scoring tool in the nonsurgical ED setting because of its simplicity. References 1. Clark C. The issues. In: Emergency Medicine, Congressional Quarterly Researcher. 1996; 6: Fakhry SM, Rutledge R, Meyer AA. Severity of illness indices. In: Weigelt JA, Lewis FR, (eds). Surgical Critical Care. Philadephia, PA: W.B. Saunders, 1996, pp 7 21.

9 1048 Olsson and Lind d REMS AND APACHE II 3. Watts CM, Knaus WA. The case for using objective scoring systems to predict intensive care unit outcome. Crit Care Clin. 1994; 10: Atkinson S, Bihari D, Smithies M, Daly K, Mason R, McColl I. Identification of futility in intensive care. Lancet. 1994; 344: Thibault GS, Mulley AG, Barnett GO, Goldstein RL, Reder VA, Sherman EL. Medical intensive care: indications, interventions, and outcomes. N Engl J Med. 1980; 302: Osler T, Horne L. Quality assurance in the surgical intensive care unit: where it came and where it s going. Surg Clin North Am. 1991; 71: Civetta JM, Hudson-Civetta JA, Nelson LD. Evaluation of APACHE II for cost containment and quality assurance. Ann Surg. 1990; 212: Sirio CL, Tajami K, Tase C, Knaus WA, Wagnher DP, Hirasawa H. An initial comparison of intensive care in Japan and the United States. Crit Care Med. 1992; 20: Hyzy RC. ICU scoring and clinical decision making. Chest. 1995; 107: Boyd CR, Tolson MA, Copes WS. Evaluating trauma care: the TRISS method. J Trauma. 1987; 27: Champion HR, Copes WS, Sacco WJ, et al. A new characterization of injury severity. J Trauma. 1990; 30: Champion HR, Sacco WJ, Copes WS, Gann DS, Gennarelli TA, Flanagan ME. A revision of the trauma score. J Trauma. 1989; 29: Baker SP, O Neill B, Haddon W Jr, Long WB. The Injury Severity Score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma. 1974; 14: Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985; 13: Porath A, Eldar N, Harman-Bohem I, Gurman G. Evaluation of the APACHE II scoring system in an Israeli intensive care unit. Isr J Med Sci. 1994; 30: Wilairatana P, Noan NS, Chinprasatsak S, Prodeengam K, Kityaporn D, Looareesuwan S. Scoring systems for predicting outcomes of critically ill patients in northeastern Thailand. Southeast Asian J Trop Med Public Health. 1995; 26: Rowan KM, Kerr JH, Major E, McPherson K, Short A, Vessey MP. Intensive Care Society s Acute Physiology And Chronic Health Evaluation (APACHE II) study in Britain and Ireland: a prospective, multicentre, cohort study comparing two methods for predicting outcome for adult intensive care patients. Crit Care Med. 1994; 22: Wong DT, Crofts SL, Gomez M, McGuire GP, Byrick RJ. Evaluation of predictive ability of APACHE II system and hospital outcome in Canadian intensive care unit patients. Crit Care Med. 1995; 23: Giangiuliani G, Mancini A, Gui D. Validation of a severity of illness score (APACHEII) in a surgical intensive care unit. Intensive Care Med. 1989; 15: Bohnen JMA, Mustard RA, Oxholm SE, Schouten BD. APACHE II score and abdominal sepsis. A prospective study. Arch Surg. 1988; 123: Poenaru D, Christou NV. Clinical outcome of seriously ill surgical patients with intra-abdominal infection depends on both physiologic (APACHE II score) and immunological (DHT score) alterations. Ann Surg. 1991; 213: Bosscha K, Reijnders K, Hulstaert PF, Algra A, van der Werken C. Prognostic scoring systems to predict outcome in peritonitis and intra-abdominal sepsis. Br J Surg. 1997; 84: Berger MM, Marazzi A, Freeman J, Chiolero R. Evaluation of the consistency of Acute Physiology And Chronic Health Evaluation (APACHE II) scoring in a surgical intensive care unit. Crit Care Med. 1992; 20: Rhee K, Fisher C, Willitis N. The Rapid Acute Physiology Score. Am J Emerg Med. 1987; 5: Efron B. Bootstrap methods: another look at the jack-knife. Annals of Statistics. 1979; 7: Hanley J, McNeil B. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143: Hanley J, McNeil B. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983; 148: Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982; 115: Lemeshow S, Teres D, Pastides H, Avrunin JS, Steingrub JS. A method for predicting survival and mortality of ICU patients using objectively derived weights. Crit Care Med. 1985; 13: Meade MO, Cook DJ. A critical appraisal and systematic review of illness severity scoring systems in the intensive care unit. Curr Opin Crit Care. 1995; 1: Antman EM, Cohen M, Bernink PJ, et al. The TIMI risk score for unstable angina/non-st elevation MI: a method for prognostication and therapeutic decision-making, JAMA. 2000; 284: Lindahl B, Toss H, Siegbahn A, Venge P, Wallentin L. Markers of myocardial damage and inflammation and long-term cardiac mortality in unstable coronary artery disease. N Eng, J Med. 2000; 343: Fiorelli M, Alperovitch A, Argentino C, et al. Prediction of long-term outcome in the early hours following acute ischemic stroke. Arch Neurol. 1995; 52: Rodrigo G, Rodrigo C. A new index for early prediction of hospitalization in patients with acute asthma. Am J Emerg Med. 1997; 15(1): Knaus WA, Rauss A, Alperovitch A, et al. Do objective estimates of chances for survival influence decision to withhold or withdraw treatment? Med Decis Making. 1990; 10: Knaus WA, Wagner DP, Lynn J. Short-term mortality predictions for critically ill hospitalized adults: science and ethics. Science. 1991; 254: Faber-Langendon K, Bartels BM. Process of forgoing life-sustaining treatment in university hospital: an empirical study. Crit Care Med. 1992; 20:570 7.