Project APOLLO/W2/AISE

Transcription

1 Project APOLLO/W2/AISE Final Report on Injury Severity Assessment Module Validation of the new ICD9-AIS extended method and calculation of patients injury severity in Europe. Alessio Pitidis, Giuseppe Balducci Italian National Institute of Health - Department of Environment and Primary Prevention; Osvaldo Chiara, Stefania Cimbanassi General Hospital Niguarda Ca Granda Milan - Trauma Team; Stefano Calderale, Sergio Ribaldi General Hospital Umberto I Rome - Emergency Department; Maria Segui-Gomez, Rafael Heredero University of Navarra Department of Preventive Medicine and Public Health; Marco Dalmasso, Carlo Mamo Piedmont Region Inter-Territorial Epidemiologic Observatory; Gabriella Poli, Carlo Olivari Liguria Region - Health Information System; Adriano Murgano Abruzzo Region Epidemiologic Observatory; Rome November 25th 2008

2 Final Report AIM The final phase of the injury severity assessment module have been dedicated to the preparation of: 1) Review Report on the state of the art in injury severity assessment methods; 2) Selection, among different methods based on anatomical or physiological (or their combination) measures of the patient health status, of those indicators more efficient in the prediction of an outcome measure (i.e. lethality rate) and applicable with less information cost to the existing clinical documents or hospital information systems. 3) Validation of the new ICD9-AIS extended method and calculation of patients injury severity in Europe by mean of developing an automatic software application for severity assessment on current data extracted from Hospital Discharge Register databases or from other ICD-9/ICD-9- CM databases. Our analysis starts from the consideration that a precise injury classification by type and severity is fundamental to the study of trauma relevance, diffusion and determinants, from both statistical/epidemiological and clinical point of view. Furthermore, classification of injuries and individual assessment of trauma patients by severity is central to the research of injured patients outcome evaluation. 1

3 THE STATE OF THE ART The first phase of the injury severity assessment module have been dedicated to the drafting of a review report on the state of the art of trauma severity assessment methods. Our analysis starts from the consideration that a precise description of the injury and its damages is necessary for clinical and epidemiological aims, more in general we can name them evaluation purposes. The need for a common language in trauma description has led through the years to the development of trauma dictionaries based on two fundamental language approaches: specific for trauma and non-specific. In the first category we can include the Abbreviated Injury Scale (AIS) method and the assessment methodologies deriving from it, in the second group we can include the methods based on the International Classification of Diseases (ICD) system. The construction of a common dictionary for trauma description fulfils two fundamental needs, on one hand it consents to describe accurately and unequivocally a determined trauma typology, on the other hand it defines an information base for the development of injury severity indicators. This last item in particular interests us in our analysis. In the AIS system the severity score attribution is based on the agreement among groups of experts. This severity scale is based essentially on the opinion of the clinicians and other experts that developed it. It can be defined, then, a subjective iso-severity patient classification system. Nevertheless, theoretically such an indicator should be an objective measure of a well determined outcome such as the survival probability or the disability level. It should be an indicator predictive of outcome expressed as individual or population health status measure (i.e. mortality or morbidity probability ratio) or health care productivity measure (i.e. inpatient length of stay or average cost). Concerning clinical outcome an injury can be described on two basic dimensions: anatomical damage and physiological consequences. Using a simplified scheme, the main variables determining the outcome can be included, as Osler indicated, in the following model: OUTCOME = f(a, F, H) (1) 2

4 where A = ANATOMICAL DAMAGE F = PHYSIOLOGICAL DAMAGE H = GENERAL HEALTH STATUS The concrete functional form of the model depends obviously on the measures effectively used. The outcome more immediate measure is the survival (death) probability. Physiological consequences of trauma can be measured by dedicated scales such as Revised Trauma Score (RTS). The anatomical damage can be measured i.e. by the AIS method and its extension to the whole patient: the Injury Severity Score (ISS). With regard to the pre-existing health status of the patient a proxy variable generally used is the age of the patient, in the hypothesis that that it is a fundamental component of his general health status. Until now we have discussed about objective or subjective methods expressed by mean of indexes of scales considering the validity of such indicator in terms of prediction capacity of the outcome. We investigated the relationship between severity indicator (i.e. ISS score) and outcome measure (i.e. mortality rate), but the trauma severity level can be directly expressed by mean of an outcome measure. As examples can be indicated at least two methods measuring the severity of trauma directly by mean of a survival probability. The first one is the TRISS method (TRauma score, Injury Severity Score, age combination index) that combines all the outcome determinants considered in the above indicated model: ISS as anatomic measure, RTS for physiological damage, age as proxy of the general health status of the patient (physiological reserve). The basic idea of this indicator is using simultaneously health status measures describing different aspects of the damage for having a greater accuracy in outcome prediction than the one obtainable by measuring each single dimension alone. In the TRISS method you start from anatomical and physiological measures of trauma severity for estimating their relationship with the survival probability in a determined population and then using that probability as measure of global injury severity. Although survival probability can be used in itself for directly grading the different levels of severity of a determined pathology without necessarily calculating its relationship with 3

5 anatomical and physiological damages. That is the Rutledge idea for ICISS (International Classification of diseases-9 Injury Severity Score) indicator that assesses the trauma patient severity as product of the survival probabilities calculated for each ICD-9-CM diagnosis as survival ratio observed in a wide study population. REVIEW OF THE STATE OF ART IN INJURY SEVERITY ASSESSMENT METHODS INTRODUCTION In the period from the early 70s up to the late 90s an increasing need for a common language in trauma description has internationally led the researchers toward the development of trauma dictionaries and methodologies useful for an accurate assessment of trauma severity based on two fundamental language approaches: specific for trauma and non-specific. In the first category we can include the Abbreviated Injury Scale (AIS) 1 method and the assessment methodologies (e.g., ISS, NISS, etc.) deriving from it as well as the assessment methods related to physiological descriptors (e.g., RTS). The second category is represented by methods such as ICISS 2, based on universal languages used for the description of every kind of pathology e.g. the International Classification of Diseases 3 system. INJURY SEVERITY Growth over the past two decades in the number and variety of injury severity scales reflects recognition that severity classification is critical for surveillance, epidemiological 4

6 investigations and evaluations of programs and policies aimed at mitigating the impact of injury at both the individual and social levels 4,5. Injury severity generally describes the impact of an injury in terms of the extent of tissue damage (that is, the pathologic evidence of trauma) and/or the physiologic response of the body to that damage. Host factors are known to mediate the effect of injury severity on outcome, such as: age, gender, and the presence of certain pre-existing conditions, including intoxicants. Since many of the early indices were initially developed for either patient triage or for use as covariates in evaluating the impact of services or systems on patient survival, these indices defined severity in terms of a "threat to life" criterion. Given the increased survival rates of trauma patients due to advances in injury control and emergency medical services, the scope of evaluation research has been broadened to include studies in which functional outcomes and disability are of increasing interest. As discussed below, current initiatives in severity measurement are focused on developing indices that measure these non-fatal outcomes. Interest has also been directed towards evaluating currently available indices (or developing new ones) for predicting consumption of health care resources and the associated costs. Here in after a detailed analysis of the main adopted trauma severity assessment systems specific for trauma or non specific. A. TRAUMA-SPECIFIC APPROACHES 1. Severity Scoring Systems Based On Anatomic Descriptors a) Abbreviated Injury Scale (AIS) The Abbreviated Injury Scale (AIS) was introduced in and is in its 5th revision 6. It is a specialized trauma classification of injuries based mainly on anatomical descriptors of the tissue damage caused by the injury. It was originally developed for use by multidisciplinary vehicular crash investigators in the 1970s as a standardized injury severity assessment tool. Subsequently, it gained acceptance with the trauma community for clinical research application. Currently, it is generally applied directly to records in 5

7 specialized trauma registries. To this day, a large AIS-user population consists of nonclinical specialists in government (other than health agencies), academia and the automotive industry. The AIS classification system was designed to distinguish between types of trauma of clinical importance as well as types of trauma of interest to vehicle designers and research engineers. It has been shown to provide a good basis for valid measurement of probability of death. The revisions to the classification of the injuries were also designed to provide a basis to support severity measures framed in terms other than probability of death, for example, probability of disability or functional impairment. However, to date, there has been little empirical data to support the notion that AIS does well in terms of probability of disability. Predicting functional impairment or disability was an important consideration in the 2005 Revision of the AIS classification system 6. The AIS has two components: (1) the injury descriptor (often referred to as the pre-dot code) which is a unique numerical identifier for each injury description; and (2) the severity score (can be referred to as the post-dot code). The severity score ranges from 1 (relatively minor) to 6 (currently untreatable), and is assigned to each injury descriptor. The severity scores are consensus assessments assigned by a group of experts and implicitly based on four criteria: threat to life, permanent impairment, treatment period, and energy dissipation. AIS values are considered to be well-ordered within body region, while there are documented discrepancies of the symmetry in scores between body regions (e.g. an AIS of 4 to the head may not be equal to an AIS of 4 to the extremities) 7. Injuries should be coded to the AIS by trained staff of trauma services or by specialists in injury data collection. Assigning AIS scores based on medical records is time consuming; it is not routinely done outside of trauma centers. The AIS is a severity measure for a single injury. Many derivative severity scales have been developed to combine multiple injuries to create a single composite score for each patient including: Injury Severity Score (ISS) 8, The New Injury Severity Score (NISS) 9, the Anatomic Profile (AP) 10 and the Anatomic Profile Scale (APS) 11. AIS is the most widely used severity score based on anatomic descriptors of injury; it is the official injury data collection tool of the National Highway Traffic Safety Administration (NHTSA) crash investigation teams. Although the reliability and validity of the AIS (and its derivatives for assessing the combined effect of multiple injuries) have been 6

8 demonstrated, their utility for population-based research is limited due to the time it takes to assign adequate scores 12. b) Injury Severity Score (ISS) Until recently, the Injury Severity Score (ISS) 13 was the accepted method of classifying anatomic injury severity in cases of multiple injury. The ISS is derived from the Abbreviated Injury Scale (AIS) 1, which assigns a severity score to an injury on a scale from 1 (minor) to 6 (major). An AIS 1 injury will, generally, not require inpatient hospital treatment, whereas an AIS 6 injury is almost certainly fatal. The ISS score takes values from 0 to 75. If an injury is assigned an AIS of 6 (unsurvivable injury), the ISS score is automatically assigned to 75. The ISS score is virtually the only anatomical scoring system in use and seemed to correlate linearly with mortality and morbidity rates, hospital stay and other measures of severity 8. It's weaknesses are that any error in AIS scoring increases the ISS error, many different injury patterns can yield the same ISS score and injuries to different body regions are not weighted. Also, as a full description of patient injuries is not known prior to full investigation and operation, the ISS (along with other anatomical scoring systems) is not useful as a triage tool. For that and other reasons, in more recent years, a number of other AIS-based measures have been suggested as alternatives to the ISS. These include the New Injury Severity Score (NISS) 9, introduced in 1997; the modified Anatomic Profile (map) 10 and the Anatomic Profile Score (APS) 11. It has been suggested that these measures better predict the probability of a patient surviving than does the ISS. The AIS classification, which forms the basis of all of these approaches, requires a manual review of the record of each patient, which is estimated to take 10 to 20 minutes 12. This makes it prohibitively expensive for the coding of injury severity in large datasets. MacKenzie et al. made possible the application of AIS coding to large datasets by developing the software package ICDMAP-85 12, released in This provided a map for conversion of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) 14 diagnosis codes to AIS codes. Therefore, ISS could be calculated simply by knowing the ICD diagnosis codes of a patient s injuries. A revised version, ICDMAP-90 15, was released in 1997 after changes to the AIS 16. To differentiate between injury severity measures based on AIS codes derived using ICDMAP and those coded by hand, the ICDMAP-based measures have been given the prefix ICD/ (e.g., ICD/ISS when ISS is based on AIS codes from ICDMAP). 7

9 c. New Injury Severity Score (NISS) Recently, researchers have proposed a new injury severity score (NISS) 9 which, unlike the ISS, considers the three most severe injuries, regardless of body region. The NISS is computed as the simple sum of squares of the three most severe AIS (1990 revision) 16 injuries. To date, two studies have reported that the NISS is more predictive of survival and performs better, statistically, than the ISS 9,17. d. modified Anatomic Profile (map) The map has four separate characterizations of the patient s injury severity: the maximum AIS value and the square root of the summed squares of the AIS values for all serious injuries (AIS 3) in each of three body regions (head/brain and spinal cord, thorax and front of neck, all other) 11. e. Anatomic Profile Score (APS) The APS is a weighted sum of the four map components, defined as APS _ (A) _ (B) _ (C) _ (maximum AIS). These four weights come from data from level 1 trauma centers participating in the Major Trauma Outcome Study Severity Scoring Systems Based On Physiological Descriptors a. Revised Trauma Score (RTS) The Revised Trauma Score (RTS) is a physiological scoring system, with high inter-rater reliability and demonstrated accurracy in predicting death. It is scored from the first set of data obtained on the patient, and consists of Glasgow Coma Scale, Systolic Blood Pressure and Respiratory Rate. Glasgow Coma Scale Systolic Blood Pressure Respiratory Rate Coded Value RTS (GCS) (SBP) (RR) = > > GCS SBP RR 8

10 Values for the RTS are in the range 0 to The RTS is heavily weighted towards the Glasgow Coma Scale to compensate for major head injury without multisystem injury or major physiological changes. A threshold of RTS < 4 has been proposed to identify those patients who should be treated in a trauma centre, although this value may be somewhat low. The RTS correlates well with the probability of survival 19 : 9

11 B. NON SPECIFIC APPROACHES 1. International Classification of Diseases (ICD) The International Classification of Diseases (ICD) is a general purpose classification of diagnoses and related matters for all health conditions and includes diagnostic codes for both nature of injury and external causes of injury. The ICD does not incorporate an explicit severity dimension. The ICD is widely used to classify health conditions in the clinical, administrative, public health promotion, and research settings in the U.S. and abroad. The Barell Matrix is a framework for ICD- 9 CM injury codes by anatomic region and type, but does not consider relative severity 20. In the United States of America (USA), mortality data are currently coded using the 10th revision of the ICD, ICD and morbidity data are currently coded using the clinical modification of the 9th revision of the ICD, ICD-9-CM 14. In many European countries (i.e. ltaly and other southern European countries) mortality data are yet coded using the 9th revision of the ICD, while morbidity data are coded using the Clinical Modification of 9th revision of the ICD (ICD-9 CM). Due to its widespread use, there has long been interest in basing the assessment of injury severity on the ICD taxonomy. 2. ICDMAP One approach for using the ICD for severity assessment has been the development of software called ICDMAP that translates ICD-9 CM coded discharge diagnoses into AIS pre-dot codes, injury descriptors, and severity scores 15. The software then uses the AIS severity scores to compute ISS, NISS, the components of the AP and the APS for the injured individual. The mapping does result in some loss of information due to differences in the injury classification systems. Resulting severity scores referred to as ICD/AIS scores are considered to be conservative measures of injury severity. ICDMAP has been validated and shown to be useful in categorizing the severity of injuries when only ICD rubrics are available 7,12,22. However, ICDMAP is becoming out of date; there is no map available for the updated ICD-9 CM, (or ICD-10) codes and there is no map to the AIS2005. Both the AIS and ICDMAP are proprietary. This poses a major barrier for many potential users. There is a strong and growing consensus that both should be non-proprietary. Furthermore, and often because of the proprietary nature of the AIS and its map to ICD, researchers have been developing empirically derived measures of severity based on ICD 10

12 diagnoses and short term outcomes such as hospital mortality coded with ICD-9 CM. Although the development of such measures was attempted as early as the 1970s 23,24, not until recently have we had sufficiently large enough databases and the statistical tools to develop and validate this approach to severity classification. 3. Measure Of The Severity Of Trauma Directly By Mean Of A Survival Probability Until now we have discussed about objective or subjective methods expressed by mean of indexes of scales considering the validity of such indicator in terms of predictive capacity of the outcome. We investigated the relationship between severity indicator (i.e. ISS score) and outcome measure (i.e. mortality rate), but the trauma severity level can be directly expressed by mean of an outcome measure. As an example can be indicated at least two methods measuring the severity of trauma directly by mean of a survival probability. a) Trauma Score - Injury Severity Score (TRISS) The first one is the TRISS method (TRauma score, Injury Severity Score, age combination index) that combines all the outcome determinants considered in the above indicated model: ISS as anatomic measure, RTS for physiological damage, age as proxy of the general health status of the patient (physiological reserve). The basic idea of this indicator is using simultaneously health status measures describing different aspects of the damage for having a greater accuracy in outcome prediction than each single dimension measure alone. In the TRISS method you start from anatomical and physiological measures of trauma severity for estimating their relationship with the survival probability in a determined population and then using that probability as measure of global injury severity. TRISS 25 determines the probability of survival (Ps) of a patient from the ISS and RTS using the following formulae: Where 'b' is calculated from: The coefficients b 0 - b 3 are derived from multiple regression analysis of the Major Trauma Outcome Study (MTOS) 15 database. AgeIndex is 0 if the patient is below 54 years of age or 1 if 55 years and over. b 0 to b 3 are coefficients which are different for blunt and 11

13 penetrating trauma. If the patient is less than 15, the blunt coefficients are used regardless of mechanism. The TRISS determines Blunt Penetrating B B calculator the probability of survival B from the ISS, RTS B and patient's age. ISS and RTS scores can be inputted independently or calculated from their base parameters. b) ICD-9 Injury Severity Score (ICISS) One family of empirically derived measures of injury severity based on the ICD is the ICD-9 Injury Severity Score, referred to as ICISS. The development of this approach to injury severity assessment is on-going and shows great promise. ICISS, first proposed by Osler and colleagues using the North Carolina Hospital Discharge Data, is based on the calculation of survival risk ratios (SRRs) for each ICD-9 CM code 2,26. The SRR s are derived by dividing the number of patients that survive a given ICD-9 CM injury diagnosis code by the number of patients with that diagnosis code. ICISS, then, is the product of the SRRs corresponding to a patient s set of injuries. ICISS was shown to outperform several important competitors, including the Injury Severity Score which is based on the AIS 11, There are several problems/issues with this early direct estimation approach that have been addressed: 1) SRRs are database-specific and it cannot be assumed that SRRs from one source will sustain accuracy and prediction in other sets of patients; 2) SRRs calculated in this manner are not independent and are contaminated by the presence of outcomes from other injuries in multiple trauma cases; 3) Interactions of injuries, which are known to exist, are not accounted for. That is, in some cases, two injuries may combine synergistically to produce a greater effect than the combination of the two acting independently. [This is equally likely to be a problem for scoring systems based on AIS and ICD.] To address some of these concerns, Meredith et al. calculated SRR s from the National Trauma Data Bank and highlighted differences between these SRR s and the original set from NCHDD, underscoring the need for comparability when using SRR s from different sources 28 Other sets of SRR s have since been calculated from various trauma registries. Also, Meredith et al. calculated and described a set of independent SRRs that were derived from patients with only a single injury 29. These independent SRR s were shown to be better predictors of survival than their traditional counterparts. The accurate measurement and incorporation of injury interactions into trauma severity models has been elusive, though some alternatives have been proposed

14 More recently, Kilgo et al. has shown that modelling a patient s most severe SRR rather than the full multiplicative ICISS score is more predictive of survival 32. This worst-injury approach may be superior because most trauma patients are either so minimally injured as not to pose a direct threat to life or so severely injured that they cannot be saved. These cases compose approximately 97% of all trauma cases and the worst-injury approach does very well for these patients. It could be argued that the SRR is exclusively an empirical observation of hospital survival given hospital admission, and lacks the independent descriptive value of AIS in terms of body system involvement. The SRR may therefore be less flexible for modelling outcomes other than hospital survival. All measures of prediction (ICISS as well as methods based on AIS) are affected by the large proportion of deaths that occur before hospital admission as well as the large proportion of deaths in older patients that occur after hospital discharge. These probabilities will also change over time, and may be affected by extraneous factors like utilization and service delivery. One major limitation in applying the ICISS to NCHS data is the dependence of the data on the system from which it was derived. ICISS scores could be derived from the NCHS National Hospital Discharge Survey. While the literature has shown that the ICISS can be internally valid, potential lack of external validity is often mentioned as a caveat. One study of interest to NCHS is the comparison of national level data from New Zealand and Australia showing both good external and internal validity 33. This study derived SRR s based on the Australian clinical modification of ICD10 (ICD-10-AM) 33. In the United States, experts involved in its most recent revision of the AIS were consulted during the development of the injury chapter of ICD-10 CM. This is likely to improve both the ICDMAP software (should a new version be developed) as well as the ICISS measures of injury severity. CONCLUSIONS In conclusion we have examined two fundamental approaches for injury severity assessment, the first one based on injury description languages specific for trauma, the second one referred to universal languages used for the description of every kind of pathology. The first approach is very costly for the necessary training of specialized codifying personnel, the complexity of clinical instrumental information to be registered, the volume of information to collect and elaborate. The universal languages based systems have normally a lesser information costs. 13

15 That difference has determined the development of two main evolution paths in the patient injury severity classification systems. On one side a year 1980 population study by Champion 24 calculated the real death probability for each HICDA-8 (Hospital International Classification of Diseases) codified diagnosis by mean of the analysis of correlation among lethality and ICD based severity scores. That study opened the way to systems based on routine clinical information currently registered, with trauma severity levels attributed directly on the base of observed outcomes, ICISS index is the most recent of them. Those systems have a good outcome predictive capacity and a fundamental advantage: can be applied on current existing clinical data. That intrinsic advantage has led some researches to follow a second approach in injury severity assessment, simply consisting in the development of a conversion dictionary among ICD-9 and trauma specific severity scores such us AIS. After many attempts the introduction of ICD-9-CM incremented the diagnosis description specificity consenting to Mackenzie the development of an ICD-9-CM/AIS-90 conversion table (ICDMAP-90) 15. Nevertheless because of the lesser diagnosis description specificity of ICD than AIS there is a high discordance among manually calculated and automatic (conversion table based) severity scores for abdominal internal organs and internal head, face and neck injuries (52% of non appropriately assigned scores). In our study we will make a selection of indicators to be applied to European data and try to improve existing methods. REFERENCES 1. AIS Rating the severity of tissue damage: I. The Abbreviated Injury Scale. JAMA (1971); 215: Osler T, Rutledge R et al.: "ICISS: an international classification of disease-9 based injury severity score". J Trauma (1996); 41(3): 380-6; discussion WHO (1977): International Classification of Diseases 1975 (IX Conference Revision). Geneva, World Health Organization. 4. MacKenzie EJ: "Injury severity scales: overview and directions for future research". Am J Emerg Med (1984); 2(6): Rivara FP (2001). Injury control: a guide to research and program evaluation. Cambridge; New York, Cambridge University Press. 14

16 6. Gennarelli T, Wodzin E (2005). The Abbreviated Injury Scale Des Plaines, Il, Association for the Advancement of Automotive Medicine. 7. Clark DE, Winchell RJ: "Risk adjustment for injured patients using administrative data". J Trauma (2004); 57(1): ; discussion Baker SP, O Neill B, Haddon W, Long WB: "The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care". J Trauma (1974); 14(3): Osler T, Baker SP et al.: "A modification of the injury severity score that both improves accuracy and simplifies scoring". J Trauma (1997); 43(6): 922-5; discussion Copes WS, Champion HR et al.: "Progress in characterizing anatomic injury". J Trauma (1990); 30(10): Sacco WJ, MacKenzie EJ et al.: "Comparison of alternative methods for assessing injury severity based on anatomic descriptors". J Trauma (1999); 47(3): 441-6; discussion MacKenzie EJ, Steinwachs DM, Shankar BS: Classifying severity of trauma based on hospital discharge diagnosis: validity of a ICD-9CM to AIS-85 conversion table. Med Care (1989); 27: Baker SP, O Neill B, Haddon W Jr: "The Injury Severity Score: a method for describing patients with multiple injuries and evaluating emergency care". J Trauma (1974); 14: ; 14. United States National Center For Health Statistics. The International Classification of Diseases, Ninth Revision, Clinical Modification. Vol 1. Ann Arbor, MI: Commission on Professional and Hospital Activities; MacKenzie EJ, Sacco W. ICDMAP-90: A User s Guide. Baltimore, MD. The Johns Hopkins University School of Public Health and Tri-Analytics, Inc, Association for the Advancement of Automotive Medicine. The Abbreviated Injury Scale, 1990 Revision. Des Plaines, IL: Association for the Advancement of Automotive Medicine; Brenneman FD, Boulanger BR, Mclellan BA et al.: Measuring injury severity: time for a change? J Trauma (1998); 44: Champion HR, Copes WS, Sacco WJ: Major trauma outcome study: establishing national norms for trauma care. J Trauma (1990); 30: Champion HR, Sacco WJ, Copes WS et al.: A revision of the Trauma Score. J Trauma (1989); 29: Barell V, Aharonson-Daniel L et al.: "An introduction to the Barell body region by nature of injury diagnosis matrix". Inj Prev (2002); 8(2): World Health Organization. (1992). International statistical classification of diseases and related health problems (Tenth revision). Geneva, World Health Organization. 22. Mullins RJ, Veum-Stone J et al.: "Outcome of hospitalized injured patients after institution of a trauma system in an urban area". JAMA (1994); 271(24): Levy PS, Mullner R et al.: "The estimated survival probability index of trauma severity". Health Serv Res (1978); 13(1): Champion HR, Sacco WJ et al.: "An anatomic index of injury severity." J Trauma (1980); 20(3): Boyd CR, Tolson MA, Copes WS: "Evaluating Trauma Care: The TRISS Method". J Trauma (1987); 27: Rutledge R, Hoyt DB et al.: "Comparison of the Injury Severity Score and ICD-9 diagnosis codes as predictors of outcome in injury: analysis of 44,032 patients". J Trauma (1997); 42(3): ; discussion Hannan EL, Farrell LS et al.: "Predictors of mortality in adult patients with blunt injuries in New York State: a comparison of the Trauma and Injury Severity Score (TRISS) and 15

17 the International Classification of Disease, Ninth Revision-based Injury Severity Score (ICISS)". J Trauma (1999); 47(1): Meredith JW, Kilgo PD et al.: "A fresh set of survival risk ratios derived from incidents in the National Trauma Data Bank from which the ICISS may be calculated". J Trauma (2003); 55(5): Meredith JW, Kilgo PD et al.: "Independently derived survival risk ratios yield better estimates of survival than traditional survival risk ratios when using the ICISS". J Trauma (2003); 55(5): Hannan EL, Waller CH et al.: "A Comparison Among the Abilities of Various Injury Severity Measures to Predict Mortality With and Without Accompanying Physiologic Information". J Trauma (2005); 58(2): West TA., Rivara FP et al.: "Harborview assessment for risk of mortality: an improved measure of injury severity on the basis of ICD-9-CM". J Trauma (2000); 49(3): ; discussion Kilgo PD, Osler TM et al.: "The worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring". J Trauma (2003); 55(4): ; discussion Stephenson S, Henley G et al.: "Diagnosis based injury severity scaling: investigation of a method using Australian and New Zealand hospitalisations". Inj Prev (2004); 10(6):

18 SELECTION OF EFFICIENT INDICATORS Information cost of severity assessment methods At the state of the art we can classify then two fundamental approaches for injury severity assessment, the first one based on injury description languages specific for trauma, the second one referred to universal languages used for the description of every kind of pathology. The first approach lead to the definition of iso-severity groups of patients and is very costly for the necessary training of specialized codifying personnel, the complexity of clinical instrumental information to be registered, the volume of information to collect and elaborate. Table 1. Workload and costs estimated for the implementation of ISS and ICISS systems Severity assessment process ISS ICISS Multiple trauma (up to 5 injuries) Admission Diagnosis and treatment Diagnosis ICD-9-CM coding 30 Minutes 30 Minutes Diagnosis AIS coding 30 Minutes Unnecessary RTS+Age+ISS coding for TRISS Comparison expected/observed outcome Personnel unit training AIS/ISS use 20 hours/year Unnecessary Total time estimated on 2,000 patients/ 2,000 hours/year 1,000 hours/year hospital/year Total additional costs ISS vs. ICISS on 10 hours/week 2,000 patients/ hospital/year $ 6,250,000 hospital/year Total additional costs ISS implementation $3,125,000/year in 500 trauma centers/year Total additional costs ISS implementation $18,750,000/year in,.000 ospedali/anno Source: adapted from Rutledge et al. 17

19 An estimate of those costs has been provided by Rutledge et al. (1997) 1. This example shows clearly that the information costs of a manual trauma specific classification system such as the AIS/ISS are largely greater than the ones deriving from an automated system based on universal classification of pathologies (and trauma). The universal languages based systems are expected to have a lower information cost. Severity assessment based on survival probability The above illustrated difference has determined the development of two main evolution paths in the patient injury severity classification systems. On one side in a year 1980 study Champion 2 calculated on a sample of 2,165 trauma patients the real death (survival) probability for each HICDA-8 (Hospital International Classification of Diseases) codified diagnosis by mean of the analysis of correlation among lethality and ICD based severity scores. The method was applicable to 1,154 of them and demonstrated a Pearson correlation index of 0,6 similar to the one calculated for the ISS. That appreciable result opened the way to the development of iso-severity classification systems based on the ICD codes or anyway based on routine clinical information currently registered, with trauma severity levels attributed directly on the basis of observed outcomes, ICISS index is the most recent of them. It can be seen as an evolution of the systems based on the analysis of the relation among survival probability and anatomical/physiological indicators such as the TRISS which is an extension to penetrating injuries of the Champion s method investigating the aforesaid relation on a sample of 2,000 blunt trauma patients 3. While the TRISS is still linked to anatomical and physiological measures in order to predict the survival probability, the ICISS directly lies on the ground of the observed probability of survival (death) used as indicator of the severity of trauma. 1 Rutledge R, Hoyt DB, Eastman B, Sise MJ, Velky T, Canty T, Wachtel T, Osler TM (1997): Comparison of the Injury Severity Score and ICD-9 diagnosis codes as predictor of outcome in Injury analysis of patients. The Journal of Trauma; vol 42, 3: Champion HR, Sacco WJ, Lepper RL et al. (1980): An anatomic index of injury severity. The Journal of Trauma; 20: Champion HR, Sacco WJ, Carnazzo AJ et al. (1981) : Trauma Score. Critical Care Medicine; 9:

20 Rutledge et al. 4 used the ICD-9-CM classification for coding the diagnoses and interventions, grading the severity of injuries by mean of the death (survival) probability per diagnosis/intervention group observed in a sample of 10,000 inpatients (DRR: Death Risk Ratio). The observational variable used in this model for the prediction of survival can be summarized as follows: Dependent Variable Independent Variable Survival (1=alive, 0=dead) DRR 1 main diagnosis DRR 2 second diagnosis DRR 3 third diagnosis DRR 4 main surgical procedure DRR 5 second surgical procedure DRR 6 external cause Because the outcome variable have been considered dichotomous, logistic regression methods where appropriate in order to investigate the functional relation among ICD-9 diagnoses and survival, estimating the functional parameters. The outcome has been defined as ratio between the probability of survival and the one of death: that consented the regression analysis by mean of a logit transformation for the prediction of association and predictive power of the independent variables. In formal terms: ln()= l o ln SRR k p DRR p = b + b 0 dx d (2) d =1 with k = maximum number of indipendent variables per patient (in this case k = 6) p = 1, 2,.., n and 4 Rutledge R, Fakhry S, Baker C, Oller D (1993): Injury severity grading in trauma patients: a simplified technique based upon ICD-9 coding. The Journal of Trauma; vol 35, 4:

21 + b x d d SRR p = eb0 1 + e b 0 + b d x d (3) where l o SRR p patient = survival odds = alive patients / dead patients = E(Y x) = survival probability of the p-th typology of DRR p = 1 - SRR p = 1 - E(Y x) = death probability of the p-th typology of patient x d b d b 0 = severity level of the d-th ICD-9 codes group graded by specific DRR d value of the group (independent variables) = coefficients of the independent variables = intercept For each typology of patient is defined a pattern of diagnoses and procedures (surgical and medical) each one associated with its own probability of death identifying the specific typology of patient and determining his probability of survival. The patterns and their DRRd if they are observed in sufficiently large sample not affected by selection biases, can be applied to any other population of trauma patients. The theoretical basis of this approach is frequentistic and lies on the idea that the mortality ratio for a specific trauma is stable when the socio-economic contexts observed are similar. Because the mechanism of the accident and the medical treatment of trauma are similar. In conclusion the mortality ratio observed in a large population of injured people is sufficiently stable during the time and across different territorial contexts so that it can be extrapolated to another population. The literature reports a good prediction capacity for these systems based on the observed survival probability as we evidenced in the review of the state of the art. A system based on the above illustrated concepts has been developed by Osler and Rutledge 5 it is named ICISS (International Classification of diseases-9 Injury Severity Score) and is based on the idea of working on survival probability. The development of this 5 Osler T, Rutledge R, Deis J, Bedrick E (1996): ICISS an international classification of disease-9 based Injury Severity Score. The Journal of Trauma; vol 41, 3:

22 idea started in the precedent studies by Champion et al. and Levy & Goldberg. ICISS expresses the severity of trauma as multiplicative effect of the survival probabilities calculated for each diagnosis (coded with the ICD-9-CM system) on the ground of the survival ratio observed in a large enough sample. In formal terms: ICISS = m SRR p = SRR d d = 1 (4) with 0 SRR d 1 and p = 1,.., n where d = number of diagnoses per typology of patient SRR d = alive/totale patients = E(X) = survival probability for the d-th diagnosis SRR p = survival per typology of patient p = typology of patient It is evident that the severity of trauma in this case is directly expressed in function of the survival probability per diagnosis. This indicator takes into account the ICD coded diagnoses only, excluding the medical and surgical procedures and any other clinical information. The implementation of ICISS is an on-going process, but it results are showing good promises. As we reported in the review of the state of the art, apart the results presented by its authors, the comparison studies in literature indicate it outperforms the existing alternative methods for assessing injury severity whether based on anatomic descriptors (such as ISS) or including physiologic information (such as TRISS). Nevertheless there are two main concerns affecting this indicator. The first one regards the exportability of the SRRs (Survival Risk Ratios) from the reference source to other data sets. In other words a group of SRRs observed in the reference sample could be specific of that population and not guarantee accuracy and prediction capacity if applied to other data sets. Meredith et al. 6 evidenced differences among the SRRs calculated in the National Trauma Data Bank and the correspondent ones observed in the North Carolina State Hospital 6 Meredith JW, Kilgo PD et al.: "A fresh set of survival risk ratios derived from incidents in the National Trauma Data Bank from which the ICISS may be calculated". J Trauma (2003); 55(5): Kilgo PD, Osler TM et al.: "The worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring". J Trauma (2003); 55(4): ; discussion

23 Discharge Database (NCHD), that originally was used by Osler and Rutlegde for the development of ICISS. The second one concerns the effect of multiple trauma. The proposed multiplicative model could not be appropriate to measure this effect because it does not take into account that in multiple trauma the single SRR for ICD diagnosis is not independent from the others because influenced by the outcomes of the other injuries. The interaction among injuries is implicitly considered in such multiplicative model, but not separately from the independent effect of each injury. So that that the effect of multiple trauma could be alternatively underestimated or overestimated depending from the patterns of ICD diagnoses included in the calculation of ICISS Kilgo et al. have shown that modelling a patient s most severe SRR rather than the full multiplicative ICISS score is more predictive of survival. This worst-injury approach may be superior because most trauma patients are either so minimally injured as not to pose a direct threat to life or so severely injured that they cannot be saved. These cases compose approximately 97% of all trauma cases and the worst-injury approach does very well for these patients. Finally one can argue that the consideration of diagnoses only, in the model, could not capture the effect of factors deriving not from the need of medical assistance (clinical condition of the patient), but on the contrary depending from the organization of the health care services and their utilization by the people. Despite of these concerns Rutledge et al. in a comparison study 7 showed an high correlation among ICISS scores computed on the same patients (test sample) by mean of SRRs calculated in different reference samples. The test sample was made of 44,032 patients registered during in the SDTR (San Diego Trauma Registry) while the databases in which the SSRs have been calculated were: SDTR itself, North Carolina State Hospital Discharge Database (NCHD), North Carolina Trauma Registry Database (NCTR) and Agency for Health Care Policy Research s Health Care Utilization Project Database (HCUP). The sample dimension varies from thousand cases in the trauma registries (TRs) up to millions of observations in the Hospital Discharge Registers (HDRs), Results in table 2 indicate a good level of association among the test sample and the other 7 Rutledge R, Hoyt DB et al.: "Comparison of the Injury Severity Score and ICD-9 diagnosiscodes as predictors of outcome in injury: analysis of 44,032 patients". J Trauma (1997); 42(3): ; discussion

24 external ones, in general good pairwise association is shown between two trauma registries or a trauma registry and a HDR. Consequently it seems reasonable to extrapolate SRRs from a trauma registry to another one or from a HDR to a trauma registry (the bigger the HDR the better the extrapolation). Table 2. Pearson R correlation coefficient ICISS SDTR patients (n.=44,032) period Data Base SDTR SRR NCHD SRR NCTR SRR p < 0,0001 NCHD SRR 0,5982 Yes NCTR SRR 0,9092 0,6617 Yes HCUP SRR 0,8888 0,4882 0,8260 Yes Source Osler, Rutledge et al. In general one can assume and literature shows internal validity for SRRs at limited distance of time from reference SRRs observation, while lack of external validity has been mentioned as a caveat. Nevertheless more recently a comparison study on national level data in Australia and New Zeeland has shown both good internal and external validity 8. The concern about the assumption of a multiplicative functional relation among injuries in multiple trauma and about the independence of survival probability for each injury maintains its validity. The accurate measurement and incorporation of injury interactions into trauma severity models is still elusive in the existing studies. In order to avoid potential problem of external validity we will consider the whole trauma inpatients in a determined country; for that reason we analyzed the National HDRs of Italy and Spain extracting from there all the patient having at least one injury coded in the main or secondary diagnoses. The data from these two countries will be used for calculating SRRs for ICD-9-CM codes while the data from the Netherlands and Austria will be used for ICD-10-CM. We will follow the approach of worst-injury calculating the most severe SRR per patient and comparing it with ICISS. The most severe SRR method appears us promising 8 Stephenson S, Henley G et al.: "Diagnosis based injury severity scaling: investigation of a method using Australian and New Zealand hospitalisations". Inj Prev (2004); 10(6):

25 because in over 1 million trauma inpatients observed in Italy during 2002 the 76 percent had a single diagnosis coded and 90 percent not more than two diagnoses. Furthermore in order to overcome the problems implicit in a multiplicative model such as ICISS we will use a clustering method computing a global SRR per patient directly in groups of patients having the same pattern of diagnoses (main and secondaries ). This approach provides a global survival probability for each type of patient directly by observing it, without assuming any hypothesis about the functional relation among the single diagnoses forming the pattern: not a single SRR for each diagnosis, but a total SRR for the whole pattern of diagnoses is directly calculated: SRR p computed not in multiplicative way but for the pattern as a whole. These patterns can be then them too aggregated when showing similar survival probabilities. In the Italian 2002 HDR trauma records we observed 9,621 patterns of diagnoses made of at least 5 persons each one and representing 89 percent of the whole trauma inpatients. A serious concern regards the fact that all the trauma severity assessment methods (based on survival probability and/or anatomical/physiological descriptors) in using mortality for the validation of injury scaling are affected by biases due to the large proportion of deaths that occur before hospital admission as well as the large proportion of deaths in older patients occurring after hospital discharge. We estimate that in Italy in 2002 on the total fatalities caused by trauma at least 50 percent occurred out of hospital. In order to take into control this issue we decided to include in the analysis mortality data from the death certificates, being normally ICD codes used for mortality less specific than ICD clinical modification codes used in HDR. We decided to distribute out of hospital deaths in the same proportion within each ICD-CM category, formally: SRR a = S i Si + DI i 1 ( k + 1) (5) where: k = ( Si + DI i ) i i DO i 24

26 SRR a = Survival Risk Ratio adjusted by out of hospital mortality Ì = i-th diagnosis code within the j-th diagnoses category S DI DO = number survived people = number dead people in hospital = number dead people out of hospital In conclusion the systems based on survival probability have a good prediction capacity of outcome and a fundamental advantage: can be (automatically) applied on large data sets of current existing clinical data. Severity assessment based on anatomic descriptors The afore said intrinsic advantage has led some researches to follow a second approach in injury severity assessment based on anatomic descriptors, simply consisting in the development of a conversion dictionary among ICD and trauma specific severity scores such us AIS. After many attempts the introduction of ICD-9-CM incremented the diagnosis description specificity consenting to Mackenzie 9 the development of an ICD-9-CM/AIS-85 conversion table. In order to validate the ICD based conversion AIS and ISS obtained by mean of the conversion table were compared with those calculated manually by trained coders reviewing the complete medical charts for 1,120 trauma inpatients. Almost all the observed cases were convertible (98.2 percent) and show globally a good agreement among ISS scores calculated with the two methods; mean values were 10.6 and 13.1 for ICD conversion and chart review respectively. The ISS scores assigned with the two methods were within ± 5 points for 70 percent of the cases. Grouping the ISS scores in three ranges (1-12; 13-19; 20 +), a classification often used in evaluation and research, the two methods of ISS scoring agreed in 68 percent of the cases. In a previous study 10 average 9 MacKenzie EJ, Steinwachs DM, Shankar BS: Classifying severity of trauma based on hospital discharge diagnosis: validity of a ICD-9CM to AIS-85 conversion table. Med Care (1989); 27: MacKenzie EJ, Shapiro S, Eastham J et al. : Reliability of the AIS-80. Med Care (1985); 23:

27 pairwise agreement in ISS among 15 coders reviewing the same medical charts in was 75 percent. These global results show that while the automatic conversion is not perfect, it provides reasonably good information on severity for large population-based data sets. Nevertheless, because of the lesser specificity in diagnosis description for ICD with respect to AIS, there is a high discordance among manually calculated and automatic (conversion table based) severity scores for abdominal internal organs and internal head, face and neck injuries (52 percent of non appropriately assigned scores for head and neck and 46 percent for abdomen) as you can see in table 3. Even though a significant proportion of inappropriate severity coding (10 percent on average) depends from errors in abstracting medical charts and using ICD classification. Furthermore for many head injuries in the MacKenzie ICD-AIS map were assigned ranges of AIS values for the same ICD code regarding. These ranges were retained in the final ICD-AIS conversion table to allow the user to choose the higher or lower values depending on the application. Table 3. Observed and maximum potential agreement between MAXAIS and ISS scores assigned by medical charts review versus ICD conversion table. Inpatients (n=1.120) in two Trauma Centers of Maryland Body Region N. Cases Observed agreement Maximum potential (%) agreement* (%) Head/Neck Face Thorax Abdomen Spine External/Burns Extremities Grouped ISS *Achievable if errors in abstracting were minimized Source: Mackenzie et al. 26

28 DEVELOPMENT OF AN EXTENDED AUTOMATIC SEVERITY SCORES ASSIGNMENT ALGORITHM. Among the systems for the classification of the inpatients we choose those based on the clinical condition of the patient, that as you know can be divided in two groups: a) the universal ones; b) those specific for trauma. Within the group a) we will use as afore said a severity assessment approach based directly on the observed survival probability, while for the group b) we adopt trauma specific severity indicators based on anatomic descriptors. We prefer the injury anatomic standardized description approach rather than physiological measures because the first is more easily observable and stable during the time. In other words there is more clinical agreement in the definition of anatomic damage and its changes during the natural course of trauma are more progressive and easily observable. We can describe now more precisely the aim of our study. In the model expressed in equation (1) the outcome of a trauma is a function of three factors: the anatomic injury, the physiological damage, the general health status of the patient. We use for the anatomic description the AIS/ISS method; as a proxy of the health status of the patient we can use his age which on average is a good indicator of his general conditions before the trauma. We preferred to ignore physiological measures because more difficult to observe and record, for RTS in example it is controversial the moment in which the physiological parameters have to be recorded, because they change rapidly during the course of trauma. Physiological parameters are dynamic measures by definition. In assigning AIS scores the method of reviewing complete medical charts by trained coders has a high information cost so that we decided to use the automatic ICD-AIS conversion method to be applied for the last year available in Italy and Spain to the whole national hospital discharge records with a trauma indicated as main diagnosis or within the secondary ones. In order to solve the problem of specificity for the scoring of head/neck and abdominal trauma we decided to revise the ICD-AIS conversion map. The main aim of this revision was to extend to the head/neck body region the bi-univocal correspondence between ICD codes and AIS scores: in order to improve the automation process to each ICD code has to correspond not a range, but only one AIS score. 27

29 AIS scores were assigned to all ICD-9-CM rubrics between and excluding foreign body through an orifice codes (939 to 939), traumatic complications (958) and late effects of injures (905 to 909). In the end codes resulted bi-univocal convertible. A panel of clinicians has been formed within the Departments of Orthopedics and Emergency Surgery of the General Hospital of the First Sate University of Rome, surgeons from the Trauma Team of the Niguarda General Hospital of Milan were included in the panel. This panel has taken as reference and revised the Guide Lines for Severity Assessment published by the Public Health Agency of Latium Region. We omit for space reasons the entire conversion table and indicate that except for head/neck, gastro-intestinal tract and other particular injuries - the general conversion criterion has been conservative: were multiple AIS scores were attributable to the same ICD-9-CM code the lowest score has been used if the difference in the range of scores was not greater than 1 otherwise the ICD code has been excluded from the conversion table. For head/neck, gastro-intestinal tract and other particular injuries a specific conversion algorithm has been developed. The algorithm for the choice of an AIS score (within a range) per ICD code lies on the probability to observe a specific score within the range of possible values, the score with the highest relative frequency will be chosen. This algorithm and the conversion table process have been automated by software procedures running in a MS-Windows environment. The computing of SRRs has also been included in the automatic procedures. Examples of the software user friendly interfaces are reported in the following images. 28

30 Fig. 1 The software application main panel Once validated the conversion process, the national hospital discharge records are being processed in order to be inputted in the above illustrated severity assessment software. The complete data process flow is charted in figure 4. In particular have been converted the diagnoses for inpatients having at least one diagnosis of trauma (within the range above specified) among the main or secondary diagnoses. The trauma inpatients of three Italian Regions (Piedmont, Liguria and Abruzzo) and of the whole Spain have been considered in the year Up to six diagnoses per patient have been examined for the Italian admissions and up to three for the Spanish ones. 29

31 Fig. 2 The ICD-9 diagnoses selection panel Fig. 3 The web style injury severity report: Maximum AIS, ISS, SRR 30

32 Fig. 4 AISE project data processing flow VALIDATION OF THE CONVERSION TABLE In order to validate the ICD-AIS revised conversion table a retrospective study has been conducted. Complete medical charts were examined by expert coders and AIS (ISS) scores were assigned to 1,054 inpatients in two major hospitals in the metropolitan areas of Rome and Milan. The retrospective sample was formed by 315 inpatients recorded in 1998 within the Department of Emergency Surgery and the Intensive Care Unit of the First University General Hospital in Rome, 439 cases observed during 2003 in the Departments of Orthopedics and Emergency Surgery and the Intensive Care Unit of the same hospital, 300 hospitalized patients recorded during 2005 in the Trauma Centre of the Niguarda General Hospital in Milan. The expected severity of injuries in the observed hospital units ranged from low to high severity cases in the two major metropolitan areas of Italy. The 91% of the cases in the whole sample resulted bi-univocal convertible by our algorithm, so that a single AIS code was assigned per each ICD-9-CM code. The validation of the ICD-AIS revised conversion table has being conducted comparing the AIS-ISS scores manually assigned by medical trained personnel reviewing the complete 31

33 medical charts for each patient in the sample with the scores automatically assigned by the conversion software. In order to assess the goodness of the conversion procedure we calculated the percentage of concordant maximum AIS scores assigned by body region in the validation sample, respectively by the manual coders in the chart review versus the automatic conversion procedure of ICD-9-CM diagnosis codes, based on the discharge abstract data. In the assessment of agreement (or disagreement) we assumed as reasonable the adoption of two different criteria of validating the conversion table: the one adopted by McKenzie et Al. 11 in their ICDMAP procedure and a broader one that we propose in this study. In accordance to the Mackenzie et Al. criteria, the identification of significant injuries can rely on the definition of injuries scored as AIS > 1. Once selected the injuries for which both methods agree on the presence of an injury, for instance as afore said, comparisons can be made on the agreement of maximum AIS by body region (MAXAIS) or on ISS scores per patient. We considered the ISS body regions, not the AIS ones, so that direct comparisons cannot be made for spine injuries. We prefer to use an approach that includes in calculations even the disagreement about the presence of a significant injury between the manual and automatic met. So that we include in the analysis even the effect of errors due to the different sensibility in detecting the presence of a significant injury. In other words, assigning the value equal to 0 to a diagnosis of trauma that we could not convert into an AIS score either by manual or automatic methods, even those zero values have been included in the analysis of agreement. Furthermore, based on the definition of significant injury (AIS score > 1), we introduce a limit of tolerance in conversion agreement between automatic and manual scores, the limit has been fixed within +1 and -1 difference in AIS scores assigned for the same diagnosis by the two methods. It is introduced for testing the sensitivity of the scoring procedure in varying the conversion method from manual to automatic scoring. In conclusion in the analysis of agreement between the two methods we suggest to consider explicitly the possible errors due to the sensibility of the methods, but reducing by construction the level of sensitivity, with the adoption of an interval of tolerance, in order to avoid the agreement measure being effected by slight variations along the AIS scale: variations equal to one 11 MacKenzie EJ, Steinwachs DM et Al., cited paper. 32

34 point on the AIS scale; value at which, according to Mackenzie et Al. an injury is not be considered significant. RESULTS We can anticipate that the obtained results indicate that the extension of the ICD-AIS map toward a revised conversion algorithm assigning univocal AIS scores even for head/neck and gastro-intestinal tract maintains the information validity of the conversion process, despite the extension of bi-univocal code conversion to these body parts. Applying the above discussed first criterion to the validation sample, in order to see if significant injuries are detected by both methods, we obtain the following frequency of cases in which the MAXAIS by body region did not match the definition of significant injury (AIS > 1) in either manual or automatic methods. Table 4. Percentage of cases in which there was a disagreement as to the presence of a significant injury (AIS > 1) to the Body Region. Manual vs. Automatic method (n. = 1.051) Body Region % MAXAIS manual > 1 and MAXAIS automatic < 1 % MAXAIS manual < 1 and MAXAIS automatic > 1 Head/Neck 35 2 Face 23 4 Thorax 21 1 Abdomen 29 5 Extremities 16 2 External/Burns 3 1 In comparing each inpatient by maximum AIS score we observe an agreement of the 49,7 percent of patients between the manual and automatic scoring. If we reduce the sensitivity of the test by applying a limit of tolerance of + 1 or -1 AIS scores to the difference between the two methods, the agreement rises up to 82,7 percent of cases. This assumption is acceptable in our opinion, when you consider that, by the MacKenzie et Al. definition, an injury must have a severity score greater than one point in order to be considered significant. 33

35 The same comparison can be made by body region for MAXAISs equal or greater than one obtaining the agreement results reported in table 5. These data show that our revised ICD-AIS map procedure vs. the original one performs slightly better for head/neck and face, worse for thorax, abdomen and extremities, much better for the external area. Nevertheless only the difference in proportions of agreements for the extremities ( 10 percentage points p = 0.001) and the external area / burns (+ 28 percentage points p = ) are statistically significant, given the power of samples. It as also to be considered that we included the spine injuries in the specific body area, while in the original study they were considered apart. Once adopted the above mentioned tolerance limit of 1 point of AIS score the proportions of agreement in our sample rises up to more than 80 percent in any body region. It depends on the fact that our manual coders tend to attribute superficial injuries to the specific body region instead of the external area. It is reflected in the fact that in our sample the number of cases presenting external injuries is just one third of the cases observed in the MacKenzie et Al. sample. The tolerance limit implicitly eliminates that effect, given that the external injuries are in general of minor severity. Even if we include in the analysis the cases that could not be converted by the automatic procedure, while maintaining the same tolerance limit, the agreement proportion in each body region is greater than the one observed in the original sample. At global level in our sample the ISS scores difference between the two methods ranged within ± 5 points for 64.6 percent of patients (95% C.I.: ) in comparison to the 70 percent observed by MacKenzie. Grouping the scores in the three classes used in the above indicated original study the agreement proportion range from 76.0 percent (95% C.I.: ) in the ISS group 1-12 to 41.9 percent (95% C.I.: ) in the ISS group and 39.5 percent (95% C.I.: ) for ISS greater than 20. Ranking the patients by the above indicated ISS groups the total observed agreement between chart review and (revised-extended) conversion table rises up to 72.7 percent (95% C.I.: ) in comparison to the 68 percent observed by MacKenzie et al. in their ICD-AIS conversion map. Our percent value of ISS-grouped scores agreement (72.7) between automatic and manual method is very near to the interrater average agreement (75 percent) among manual coders reviewing the same complete medical charts, used by MacKenzie et al. as reference for validating the conversion table. This agreement value could even be incremented by eliminating errors in abstracting medical charts. In the HDR 34

36 records we often observed an incomplete reporting of ICD codes in comparison to the injuries reported in the complete medical charts and scored by manual coders. Anyway having the ICD-9-CM diagnosis codes less specificity in the description of trauma than the AIS ones the agreement could never be perfect. In the following figure we observe in the validation sample that automatically assigned MaxAIS scores based on the conversion of ICD-9-CM codes from discharge abstracts tend to underestimate the severity of injury in comparison to the MaxAIS scores manually assigned by coders from charts review. It can be noticed how the automatic AIS scores frequencies are systematically higher than the manual ones for low severity scores and the contrary for the intermediate and very severe ones. Table 5. Observed agreement between MAXAIS and ISS scores assigned by medical charts review versus ICD conversion table. Inpatients (n=1.051) in two Major Hospitals of Italy Body Region MAXAIS >= 1 MAXAIS >= 0 and tolerance MAXAIS >= 1 and difference +/- 1 AIS score tolerance difference +/- 1 AIS score N. Observed N. Observed agreement Observed agreement Cases agreement (%) Cases (%) (%) Head/Neck Face Thorax Abdomen Extremities External/Burn s Grouped ISS

37 Fig. 5 In conclusion these results on the validation sample indicate that with the extension to head of unequivocal ICD-AIS mapping, instead of loosing information on the severity of injuries, we significantly improved quality of automatic versus manual scoring. That consideration lead us to the application of the automatic conversion procedure to the national population-based data from Hospital Discharge Registers. We obtained a biunivocal assignment of a single AIS score per each ICD-9-CM diagnosis on 60 percent of the Italian inpatients (62% in Piedmont, 57% in Liguria and 61% in Abruzzo) and on 50 percent of the Spanish ones. More than admissions have been considered for Spain and more than for the Italian Regions. The sex age distribution was similar in the two populations as shown in the following figures. 36

38 Fig. 6 Fig. 7 Fig. 8 The maximum AIS for each patient has been calculated; the overall MaxAIS and the ones by body region present very similar frequency distrubutions in the two Countries. For the overall MaxAIS the value 0 indicate patient with uncorvetible ICD-9-CM codes. 37

39 Fig. 9 Fig

40 Fig. 11 Fig

41 Fig. 13 Fig

42 Fig. 15 There is a strong agreement in the two Countries among the overall MaxAIS relative frequency distributions. We assigned the value 0 to the ICD-9-CM diagnoses uncovertible into bi-univocal severity scores. The agreement is also strong in consideringe the MaxAIS scores by body region. It is interesting to note how Head/Neck and Thorax show the greatest proportions of intermediate and severe injuries in the trauma inpatients of both countries. In general the Spanish hospital data show a more selective style of admission for trauma patients than the Italian ones. It is reflected in the fact that for head/neck, face and thorax body regions the Spanish cases have a significantly greater frequency than the Italians of intefrmediate severity injuries. It can be beter seen in the ISS distribution which has different shape in the two countries. In Italy there is most of the cases have low severity (more than 60% in the ISS class 0-4) whilst in Spain the main group is represented by intermediate severity inpatients (more than 80% in the ISS classes 5-9 and 10-16). 41

43 Fig. 16 In order to check the applicability of our revised ICD-AIS map procedure we applied it to 10 samples of trauma inpatients in each Country. The samples consisting in cases each one, randomly extracted from the Hospital Discharge Register (HDR) data of the examined Country. The MaxAIS relative frequency distributions in the observed samples either Spanish or Italian are more concentrated on low severity scores than the one of the validation sample, as we expected because the validation sample has been extracted respectively from the Emergency Departments of the biggest University General Hospital of Rome and the Trauma Center of Milan, the largest metropolitan areas of Italy. These hospitals are highly specialized and therefore considered second level emergency units because they attract severe cases from the other smaller hospitals in which the patient firstly arrived. The observation of the charts indicates us that the automatically converted samples (from ICD-9-CM to AIS) are internally strictly concordant as to overall (per patient) MaxAIS distribution within the same Country. In other words by examining different sets of cases, 42

44 casually selected and of the same dimension of the validation sample, we always obtain the same severity distribution for trauma inpatient within the examined Country.. Fig. 17 Furthermore the severity distributions are strongly agreeing even when the samples observed through the different Countries. Apart the difference in frequency between low (expected higher in Italy) an intermediate severity scores ((expected higher in Spain), because the style of admission. In other words our revised conversion procedure is produces results that are consistent internally (within) and externally (through) to the examined Countries 43

45 Fig. 18 In conclusion the revised procedure for automatic conversion of discharge abstract ICD-9- CM coded diagnoses into AIS scores produced valid figures in comparison with the severity scores manually assigned by trained coders, on complete medical chart review, to inpatients of the same validation sample. Furthermore it proved high consistency of the results when applied in different samples from two different Countries, so that its requirements of validity and consistency are fully complied. 44