A Concept for the Validation of Fracture Classifications

Transcription

1 SPECIAL INTEREST A Concept for the Validation of Fracture Classifications Laurent Audigé, DVM, PhD,* Mohit Bhandari, MD, MSc, FRCS(C), Beate Hanson, MD, MPH,* and James Kellam, MD Summary: The fracture classification systems currently used most frequently were not developed or validated by rigorous scientific evaluation methods. This paper discusses the classification of fractures from an epidemiological and clinical decision-making perspective and proposes a standardized methodological concept for their development and scientific validation. Classification categories are clinically relevant entities that surgeons should be able to use for diagnosis with sufficient confidence to limit misclassification and associated treatment errors. The process of validation should assess the value of specific clinical information (eg, the use of radiographs or computed tomography scans) in increasing the probability of a correct diagnosis. A 3-phase validation concept is proposed where: 1) classification categories are defined and the classification process using specific diagnostic images is evaluated by experts in a series of agreement studies (reliability, accuracy, likelihood ratios); 2) a multicenter agreement study is conducted among a representative group of future users of the classification; and 3) the classification proposal is applied in the context of a prospective clinical study to assess its clinical usefulness. Key Words: clinical epidemiology, fracture classifications, validation, agreement studies (J Orthop Trauma 2005;19: ) Injury localization and severity are important factors influencing the surgeon s choice of treatment and the patient s anatomic and functional outcome. These factors should therefore be documented in any clinical study for proper scientific evaluation of treatment interventions and outcomes. One aspect of injury very frequently recorded is the description and subsequent classification of bone trauma the fracture. Numerous fracture classification systems have been proposed in orthopaedics, 1 3 but only a small number of them have become widely accepted in practice, such as the Müller-AO classification of long bones and the Orthopaedic Trauma Association (OTA) Fracture and Dislocation Compendium. 4 6 Accepted for publication November 20, From *AO Clinical Investigation and Documentation, AO Foundation, Davos Platz, Switzerland, Department of Clinical Epidemiology and Biostatistics, McMaster University Health Sciences Centre, Hamilton, Ontario, Canada, and Carolinas Medical Center, Charlotte, NC. The authors are not supported by specific grant. Dr. Bhandari is funded, in part, by a Canada Research Chair in Musculoskeletal Trauma and Surgical Outcomes. Reprints: Laurent Audigé, DVM, PhD, AO Clinical Investigation and Documentation, AO Foundation, Stettbachstrasse 10, Dübendorf CH-8600, Switzerland ( laurent.audige@aofoundation.org). Copyright Ó 2005 by Lippincott Williams & Wilkins Classification systems have multiple purposes. They should facilitate communication between physicians and assist documentation and research. They should also have prognostic value for patients and assist physicians in planning their management. According to Maurice Müller, a classification is useful only if it considers the severity of the bone lesion and serves as a basis for treatment and for evaluation of the results. Although it is possible that the most used classification systems more or less have these attributes, this is poorly documented. Few fracture classifications are validated before they are officially accepted and promoted For example, the Neer classification system was shown to have prognostic value in predicting avascular necrosis, 14 thereby supporting its routine use in orthopaedic fracture care. However, subsequent studies reported its questionable interobserver reliability, casting doubt on previous observations of its validity Indeed, in the context of treatment decision processes, a poorly validated classification will be a biased predictor of patient outcomes, and its use may result in unnecessary harm to patients (if an aggressive treatment is given to a patient wrongly classified as being in a severe classification category). In the course of scientific clinical research, poorly validated classifications will allow the misclassification of patients and bias the study, making the comparison of patient populations between studies difficult. Therefore, it is important to judge whether a classification process measures what we want it to measure, and how well 19 before it is widely accepted into clinical practice. It this respect, 2 significant shortcomings of classification should be addressed. The first is that classification systems often transform continuous variables into categorical ones; for instance, the obliquity of diaphyseal fractures is reduced to a dichotomous variable (#30 versus.30 ) in the Müller-AO classification of long bones. 4 The second is that classification diagnoses are typically based on the assessment of radiographs or other imperfect diagnostic imaging techniques. 20 Attempts to validate existing classifications were made mainly through reliability studies that were initiated long after surgeons had started to use the classifications. Results were predominantly disappointing. 8,21 The objective of this article is to review some methodological issues related to the purpose and quality of classifying fractures and to propose a structured approach for the future development and validation of classification systems in orthopedics. CLASSIFYING FRACTURES IS A DIAGNOSTIC PROCESS Classifying a fracture is equivalent to making a diagnosis. A few definitions should be provided to help 404 J Orthop Trauma Volume 19, Number 6, July 2005

2 J Orthop Trauma Volume 19, Number 6, July 2005 Special Interest understanding the issues involved. To illustrate them, let us consider that a patient with a hip fracture attends your clinic. Upon physical examination, you suspect a fracture of the femoral neck. Classification system: the set of fracture categories and the structure that define the important fracture diagnoses to be made. For instance, the Garden classification is a wellknown classification system for femoral neck fractures. 22 The classification of femoral neck fractures as undisplaced or displaced is a clinically important fracture diagnosis because it will influence the course of treatment Another classification system as stable versus unstable fractures was recently proposed. 26 True fracture status: the fracture category to which a given fracture truly belongs. Let us consider that the hip fracture of your patient is truly displaced and that you want to know that truth, or at least to be very confident about it. Classification diagnosis (or rating): the fracture category to which a given fracture is allocated or diagnosed by an observer (eg, a surgeon). If you diagnosed a displaced femoral neck fracture, this classification diagnosis would be in agreement with the truth. However, a misclassification can occur if you incorrectly diagnosed an undisplaced fracture. Classification process: the process by which surgeons allocate fractures to respective classification categories. Such a process can be understood as a diagnostic test. It may be described in terms of several components, such as the type and quality of image modalities used, the time of classification, the type and training of observers, a possible recording tool used (eg, a software), and the diagnostic rule (eg, single or consensus diagnosis) (Table 1). For instance, the process of classifying femoral neck fractures involves the examination of standard anteroposterior and lateral radiographic views, as well as computed tomography (CT) scans. You may examine the images alone or consult another surgeon or your radiologist. Gold standard classification process and diagnosis: the term gold standard is jargon to describe a method, procedure, or measurement that is widely accepted as being the best available. In the context of classification systems, the gold standard classification diagnosis is the fracture category to which a given fracture is allocated TABLE 1. Components of a Fracture Classification Process Classification Components Type and quality of image modalities used Time of classification Type and training of observers Recording tool Diagnostic rule Examples Radiographs, CT scan, 3D reconstruction, image intensifier, etc Postinjury, at the operating table, etc Surgeon, assistant, radiologist, etc Coding by observers, use of specific softwares, etc Single rating, multiobserver agreement, majority of ratings, etc via a gold standard classification process. The gold standard classification diagnosis is usually highly correlated with the true fracture status but not necessarily perfectly; hence, this diagnosis is believed to be similar or very close to the true fracture status. The gold standard classification process for hip fractures may not be certain. However, if you classified the hip fracture in agreement with your colleagues after assessing all available clinical information (including observations during surgery), the resulting diagnosis may be the closest to the truth. The corresponding coding would be considered as the gold standard classification. Reference standard classification process and diagnosis: the application of a gold standard classification process to make the best diagnosis may not be possible either because it is too invasive, unethical, or its timing is inadequate (eg, postsurgery). A fracture may also not be examined during an open surgery, and certainly not if a percutaneous technique is used. As recommended by Knottnerus and Muris, 27 we will use the term reference standard instead of gold standard to account for this imperfection. The value of a classification system is first and foremost related to its clinical significance, in particular for making a prognosis and/or helping the treatment decision process. But before such a system can be studied and applied in practice, there must be defined a classification process that leads to classification diagnoses with sufficient accuracy. Would the classification of femoral neck fractures as undisplaced or displaced useful if you had 20% chance of misdiagnosing a truly displaced fracture? The consequences for the patients may be significant, because surgeons often treat nondisplaced fractures with internal fixation, whereas displaced fractures are currently perceived to be best managed with arthroplasty. 28 Let us consider what would happen in your clinic. When the patient with a fractured hip presents in the emergency department, you may be able to determine the most likely fracture category from clinical examination and expertise, but this prior assessment may not be conducted with high confidence (pretest probability of diagnosis). This probability may not be considered sufficient to make the final classification diagnosis that is necessary before deciding on the treatment. For instance, you would need to be at least 90% sure of your diagnosis to make a treatment decision. The performing of diagnostic tests, such as taking radiographs and/or CT scanning, provides clinical information, which improves the surgeon s confidence in a diagnosis (posttest probability). In this process, the prior probability could be very low, and the quality of the test used (diagnostic image) depends on its ability to increase the probability of a diagnosis to a level allowing a treatment decision to be made with confidence. This ability is influenced by many factors, such as the type and quality of image modalities used, as well as the experience of the reader (surgeon, radiologist, etc.), and we believe its assessment represents an important aspect of classification validation. Consequently, when developing a classification, specific rules and/or guidelines describing the classification process must be established to minimize variability and inaccuracy in classification diagnosis. q 2005 Lippincott Williams & Wilkins 405

3 Special Interest J Orthop Trauma Volume 19, Number 6, July 2005 METHODOLOGICAL STANDARDS ARE NEEDED The validation of fracture classifications involves the measurement of several important objective quality parameters. Most of this methodology was originally developed in the social sciences, 19,29 but some adaptations to the specific needs of orthopedics have been made. We have grouped validity criteria into 3 areas. First, the classification system must be based on clinically relevant diagnostic items. For instance, if a classification is expected to measure fracture severity, the diagnostic items should be somewhat related to severity, such as some specific measures of displacement, the presence of bone fragments, or the involvement of joints (issue of face validity). All important items should be considered and not left out (issue of content validity). In the resulting classification system, all possible fractures should fit 1 and just 1 category (all inclusive and mutually exclusive). When developing a classification system, we believe a consensus should be obtained from a recognized group of experts about the purpose of the system and the relevance and completeness of concerned diagnostic items. Secondly, we need to know how well the classification process performs in term of its reliability and accuracy. Reliability measures to what extent repeated applications of the classification process on the same fractures agree (the reliability of successive observations by the same rater is defined as intrarater reliability or repeatability, and the reliability of successive observations by different raters is defined as interrater reliability ). Accuracy measures how well classification diagnoses fit with the true fracture status, which is best recorded by a gold standard classification process, or at least by an acceptable reference standard. The investigation of classification accuracy is rare, as implemented in only 20% of 44 published studies. 21 Indeed, the frequent lack of an adequate reference standard classification process is a major concern. Nevertheless, we believe investigators should systematically attempt to determine the true fracture status of cases included, even if the best available method for any study remains imperfect. An unreliable classification process is unlikely to be accurate (at least some diagnoses are wrong), but a reliable classification process may not be accurate. Thus, both reliability and accuracy are key components to any fracture classification. 8 Thirdly, the classification system must have construct validity, ie, its categories must be associated with relevant patient outcomes in the context of specific fracture management plans. For instance, although a classification system may appear to measure severity, its prognostic value for patient outcomes may be only suspected in its development phase and should be appropriately documented. A classification should be considered as validated when these criteria are met. 19 Most classification systems were proposed for use in clinical practice without passing any test to determine their validity. 8,21 The consequences of this approach are that surgeons do not have the necessary scientific evidence for the usefulness of the proposed systems; they were often left alone to judge which classification process should be best applied, and any revision of the existing systems is more difficult once they are in use. Hence, there was a need for a concept allowing early validation studies and on-going necessary revisions based on scientific data. In the following, we present our view on the important methodological issues to achieve this goal. ISSUES OF CLASSIFICATION DEVELOPMENT Several authors reported that a fracture classification should be related to specific treatment options, 3,30 but like other authors 12,31 33 we do not share this view. A classification system should refer to biologic entities (eg, bone anatomy, fracture severity, etc.). In practice, a fracture classification system may be strongly related to treatment choice, because some surgeons will base their treatment decisions essentially on the classification. It is our opinion, however, that this is not a one-to-one relationship and treatment options (that may change over time) should not per se be part of the definition of classification categories. For instance, the Garden classification system usually divides patients into 2 proposed treatment categories: arthroplasty for displaced fractures (Garden III, IV) and internal fixation for undisplaced fractures (Garden I and II). 34 The choice of treatment, however, depends upon multiple factors such as known prognostic factors for specific outcomes and other factors related to the surgeon s experience, the patient s personal values, and the costs involved. Ultimately, a unified classification system should be developed to address all purposes. It would always be useful for documentation and epidemiological studies, but its usefulness for decisionmaking will depend on its prognostic value for clinically relevant outcomes. All bony biologic entities with proven prognostic value should be included in the classification if their categorization is justified. Developing a classification implies creating categories, which may be a challenge if a diagnostic item is recorded on a continuous scale. For instance, nonspiral diaphyseal fractures of the long bones are classified as oblique or transverse in the Müller-AO classification 4 because the obliquity of the fractures plays an important role in prognosis and treatment. The scale used, however, is derived from the measurement of the angle formed by the long bone axis and the fracture line, ie, a continuous parameter. Oblique fractures are defined when the angle reaches at least 30. We believe the choice of such a threshold should be evidence-based and supported by 2 main observations: 1) the threshold should clearly distinguish different types of fractures and is better justified when a small proportion of fractures are measured around the threshold; and 2) the prognostic value is not reflected by a continuous relationship between the continuous parameter and the outcomes. If there is a lack of scientific justification to categorize, the continuous parameters may serve the surgeons better if they are recorded outside the proposed classification system. Classifications based on categorized continuous parameters should always be done after effective measurement of the parameter following clear guidelines, in particular when the value is close to the chosen threshold. A research pathway should be used to validate continuous fracture measurements, including the implementation of agreement studies q 2005 Lippincott Williams & Wilkins

4 J Orthop Trauma Volume 19, Number 6, July 2005 Special Interest SEARCH FOR AN ADEQUATE REFERENCE STANDARD CLASSIFICATION PROCESS As mentioned earlier when conducting agreement studies, the best effort should be made to quantify classification accuracy, then to estimate the most likely distribution of true fracture status in the sample of fractures used via a reference standard classification process. Several methods have been described and used in the past. The easiest reference standard classification may be obtained from consensus between participating raters 13,18,38,39 or from an independent expert panel. 40 Such a consensus classification is likely to be closer to the true status than any single rating; however, it is dependent on the true ability of diagnostic images to indicate a specific classification category and on the expertise of the raters. Additional diagnostic images may be used, such as when CT scan assessments serve as a reference standard for the evaluation of radiographic assessments 41 (although CT scans may not be obtained from all patients). Alternatively, it may be that the best reference standard classification is derived from the sum of all the information collected, ie, everything from imaging studies to the findings at operative intervention. This postoperative (or posttreatment) classification would be better recorded prospectively while collecting cases for an agreement study, as a good understanding of the clinical information used is important. In the absence of an acceptable reference standard, reliance on some statistical methods may be useful, 42 as applied recently in the evaluation of a pediatric long-bone fracture classification. 43 A PROPOSED VALIDATION PATHWAY We propose a 3-phase concept that should be sequentially followed before a classification can be labeled as validated (Fig. 1). Phase 1 Study: Development or Revision of Classification Systems With Clinical Experts Phase 1 studies are better conducted by clinical experts to develop or revise classification systems. Classification proposals should be based on extensive literature review and clinical expertise. They could be largely based on existing systems that one wishes to validate. Face and content validity should prevail. Experts should provide a clear and unambiguous description of classification categories, as well as specific diagnostic items to be assessed individually, before FIGURE 1. Research and development phases for the validation of fracture classification systems. incorporation into a classification proposal. Ideally, a hypothesis as to how the classification categories should be related to clinically relevant patient outcomes and how they help in planning treatment should also be formulated, as it is required for the planning of phase 3 clinical studies. Before a classification system can be used in practice and clinical studies, it is critical to determine and assess the classification process (Table 1). It usually involves the use of imaging modalities (eg, CT scan and/or radiograph) combined with clear guidelines for their examination. The most appropriate classification process allows fracture diagnosis with an acceptable level of accuracy given the clinical context (including ethical and cost issues). They can be assessed in multisurgeon agreement studies. Pilot agreement studies (assessing both reliability and accuracy) should be applied at an early stage in the development process with a limited number of cases and observers (usually the experts themselves) so that factors associated with poor reliability and accuracy can be readily identified and the proposal modified accordingly. Most common sources of disagreement between surgeons are related to the surgeon themselves, the clinical data used for conversion into classification categories, and the conversion procedure itself. 8 Revised proposals should be tested again with similar pilot agreement studies until experts are confident that the final proposal could be tested on a wider scale. Pilot agreement data should show that the proposal is likely to be reliable and accurate enough in practice. These data justify the risk of higher costs associated with larger pragmatic multicenter agreement studies (see phase 2). Such a phase 1 evaluation process has recently been completed for a pediatric long-bone fracture classification proposal 43,44 after a series of 4 classification sessions over a 2-year period. The experts involved recognized that this would lead to a classification proposal with more clinically relevant categories, clearer definitions of categories, and a well-defined classification process. If an existing classification system requires validation, investigators should initiate the process in a phase 1 study and consider the classification system as a starting point. Experts may need to define the classification process, and pilot agreement studies will help them identify areas for improvement. Our current experience is that results from the first pilot studies are unlikely to be fully acceptable, but have the power to drive appropriate and relevant changes. Phase 2 Study: Pragmatic Multicenter Agreement Study in Clinical Practice In phase 2, a pragmatic multicenter agreement study should be implemented using a large number of representative cases and raters with various levels of expertise. Pragmatic means that the reliability and accuracy of the classification process is conducted as close as possible to the real-life daily clinical routine, instead of the well-controlled pilot experiment involving only clinical experts. Changes to improve the classification should not be excluded in this phase, as the process must remain flexible. Depending on the extent of the changes made, any part of the classification system and/or process might be assessed again via pilot agreement studies as in phase 1. q 2005 Lippincott Williams & Wilkins 407

5 Special Interest J Orthop Trauma Volume 19, Number 6, July 2005 We believe this study phase is important because experts can produce excellent results after a series of evaluations. 43 Although very promising, these results must be verified in a broader clinical context to increase their generalizability to the whole community of surgeons. Practically, this study involves the classification of fractures from diagnostic images, as did the pilot agreement studies in phase 1. When conducting agreement studies, a judgment should be made as to when results are acceptable, so that the validation process can move into the subsequent phase. This judgment should be made on the basis of estimations of classification accuracy and not only reliability parameters such as the Kappa coefficient. 21 What is acceptable should be decided specifically for each classification, as some classification categories may have more clinical importance than others. Practically, the validation process should attempt to measure the extent to which the diagnostic imaging technique used can increase the confidence that a diagnosis is correct. This evaluation should be implemented before anything can be said about the clinical usefulness of a classification. Phase 3 Study: Prospective Clinical Observational Study When a classification system has been defined and the classification process has been evaluated, clinical studies should be conducted to assess the clinical relevance and usefulness of classification categories. In phase 3, the proposed classification system can be applied in daily practice and assessed in the context of prospective clinical studies to investigate how it is related to patient outcomes given several treatment scenarios (issue of construct validity). Several studies have been conducted to verify the prognostic value of the Müller-AO long bones or AO/OTA classification systems They showed that the prognostic value of classification categories depends on the targeted outcomes and that factors influencing these outcomes are likely to be multiple and interrelated. Bhandari et al 48 showed that transverse, oblique, and segmental tibial shaft fractures treated with intramedullary nailing had a significantly higher risk of reoperation than spiral fractures, but only transverse fractures appear to have a prognostic value when other factors such as open versus closed injury and presence of less than 50 percent of cortical contact were considered in the analysis. Therefore, valid classification categories should not be assessed in isolation using univariable statistics. A multivariable approach is necessary, and potential confounding factors should be investigated with care. In this phase, a large observational study should be conducted with accurate recording of most clinically relevant outcomes and their respective known or suspected prognostic factors, including treatment options. This study design permits the assessment of the prognostic value of the classification while considering the effect of other known prognostic factors. Investigators need to determine as accurately as possible the true status of the fractures under assessment (using all available clinical information as mentioned earlier), because misclassification of fractures is likely to bias (usually underestimate) their true prognostic value. This last phase is probably the most difficult of all, because prognostic information will vary with treatment applied, and treatment will change over time. Ongoing monitoring of register-type observational databases should be considered. CONCLUSION In this paper, we reviewed methodological issues related to the classification of fractures, and the several validation criteria to be addressed. Methodological standards are needed, and so is empirical research to identify important practical methodological issues. We propose a 3-phase approach to the development and validation of fracture classification systems. If accepted, this process should limit the proliferation of useless and poorly accepted classifications and help to obtain a general consensus for classifying fractures in orthopaedics. We recommend that one phase should be completed before the next one is started so that flexibility for changes remains before a final classification is promoted. Proposed revisions to existing classifications should follow the same path of evaluation. The decision to move into the next phase can be made upon evaluation of results using objective criteria. This approach will allow evaluation of whether a classification process measures what we want it to measure and how well. REFERENCES 1. Rockwood CA, Green DP, Bucholz RW, et al. Rockwood and Green s Fractures in Adults. Philadelphia, PA: Lippincott-Raven; Browner BD, Jupiter JB, Levine AM, et al. Skeletal Trauma Fractures, Dislocations, Ligamentous Injuries. Philadelphia, PA: W.B. Saunders; Bernstein J, Monaghan BA, Silber JS, et al. Taxonomy and treatment a classification of fracture classifications. J Bone Joint Surg Br. 1997; 79: Müller ME, Nazarian S, Koch P, et al. The Comprehensive Classification of Fractures of Long Bones. Berlin: Springer-Verlag; OTA. Fracture and dislocation compendium. Orthopaedic Trauma Association Committee for Coding and Classification. J Orthop Trauma. 1996;10(suppl 1):v Murphy WM, Leu D. Fracture classification: biological significance. In: Colton CL, Fernandez Dell Oca A, Holz U, et al, eds. AO Principles of Fracture Management. New York, NY: Thieme; 2000: Lindsjo U. Classification of ankle fractures: the Lauge-Hansen or AO system? Clin Orthop. 1985: Garbuz DS, Masri BA, Esdaile J, et al. Classification systems in orthopaedics. J Am Acad Orthop Surg. 2002;10: Ackermann C, Lam Q, Linder P, et al. [Problems in classification of fractures of the proximal humerus]. Z Unfallchir Versicherungsmed Berufskr. 1986;79: Colton CL. Telling the bones. J Bone Joint Surg Br. 1991;73: Burstein AH. Fracture classification systems: do they work and are they useful? J Bone Joint Surg Am. 1993;75: Martin JS, Marsh JL. Current classification of fractures. Rationale and utility. Radiol Clin North Am. 1997;35: Brady OH, Garbuz DS, Masri BA, et al. The reliability and validity of the Vancouver classification of femoral fractures after hip replacement. J Arthroplasty. 2000;15: Seemann WR, Siebler G, Rupp HG. A new classification of proximal humeral fractures. Eur J Radiol. 1986;6: Sidor ML, Zuckerman JD, Lyon T, et al. The Neer classification system for proximal humeral fractures. An assessment of interobserver reliability and intraobserver reproducibility. J Bone Joint Surg Am. 1993;75: Siebenrock KA, Gerber C. The reproducibility of classification of fractures of the proximal end of the humerus. J Bone Joint Surg Am. 1993;75: Brien H, Noftall F, MacMaster S, et al. Neer s classification system: a critical appraisal. J Trauma. 1995;38: Bernstein J, Adler LM, Blank JE, et al. Evaluation of the Neer system of classification of proximal humeral fractures with computerized 408 q 2005 Lippincott Williams & Wilkins

6 J Orthop Trauma Volume 19, Number 6, July 2005 Special Interest tomographic scans and plain radiographs. J Bone Joint Surg Am. 1996; 78: Bland JM, Altman DG. Statistics notes: validating scales and indexes. Br Med J. 2002;324: Martin J, Marsh JL, Nepola JV, et al. Radiographic fracture assessments: which ones can we reliably make? J Orthop Trauma. 2000;14: Audigé L, Bhandari M, Kellam J. How reliable are reliability studies of fracture classifications? A systematic review of their methodologies. Acta Orthop Scand. 2004;75: Garden RS. Low-angle fixation in fractures of the femoral neck. J Bone Joint Surg Br. 1961;43B: Thomsen NO, Jensen CM, Skovgaard N, et al. Observer variation in the radiographic classification of fractures of the neck of the femur using Garden s system. Int Orthop. 1996;20: Eliasson P, Hansson LI, Karrholm J. Displacement in femoral neck fractures. A numerical analysis of 200 fractures. Acta Orthop Scand. 1988;59: Cree M, Yang Q, Scharfenberger A, et al. Variations in treatment of femoral neck fractures in Alberta. Can J Surg. 2002;45: Beimers L, Kreder HJ, Berry GK, et al. Subcapital hip fractures: the Garden classification should be replaced, not collapsed. Can J Surg. 2002;45: Knottnerus JA, Muris JW. Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol. 2003;56: Bhandari M, Devereaux PJ, Swiontkowski MF, et al. Internal fixation compared with arthroplasty for displaced fractures of the femoral neck. A meta-analysis. J Bone Joint Surg Am. 2003;85A: Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to Their Development and Use. New York, NY: Oxford University Press; Chan PS, Klimkiewicz JJ, Luchetti WT, et al. Impact of CT scan on treatment plan and fracture classification of tibial plateau fractures. J Orthop Trauma. 1997;11: Newey ML, Ricketts D, Roberts L. The AO classification of long bone fractures: an early study of its use in clinical practice. Injury. 1993;24: Kreder HJ, Hanel DP, McKee M, et al. Consistency of AO fracture classification for the distal radius. J Bone Joint Surg Br. 1996;78: Colton CL. Fracture classification a response to Bernstein et al. J Bone Joint Surg Br. 1997;79B: Oakes DA, Jackson KR, Davies MR, et al. The impact of the garden classification on proposed operative treatment. Clin Orthop. 2003: Leung YL, Beredjiklian PK, Monaghan BA, et al. Radiographic assessment of small finger metacarpal neck fractures. J Hand Surg [Am]. 2002;27: Bozentka DJ, Beredjiklian PK, Westawski D, et al. Digital radiographs in the assessment of distal radius fracture parameters. Clin Orthop. 2002: Cole RJ, Bindra RR, Evanoff BA, et al. Radiographic evaluation of osseous displacement following intra-articular fractures of the distal radius: reliability of plain radiography versus computed tomography. J Hand Surg [Am]. 1997;22: Oskam J, Kingma J, Klasen HJ. Interrater reliability for the basic categories of the AO/ASIF s system as a frame of reference for classifying distal radial fractures. Percept Mot Skills. 2001;92: Johnstone DJ, Radford WJ, Parnell EJ. Interobserver variation using the AO/ASIF classification of long bone fractures. Injury. 1993;24: Tiel-van Buul MM, van Beek EJ, van Dongen A, et al. The reliability of the 3-phase bone scan in suspected scaphoid fracture: an inter- and intraobserver variability analysis. Eur J Nucl Med. 1992;19: Borrelli J Jr, Goldfarb C, Catalano L, et al. Assessment of articular fragment displacement in acetabular fractures: a comparison of computerized tomography and plain radiographs. J Orthop Trauma. 2002; 16: Hagenaars J, McCutcheon A, eds. Applied Latent Class Analysis. Cambridge: Cambridge University Press; Audigé L, Hunter J, Weinberg A, et al. Development and evaluation process of a paediatric long-bone fracture classification proposal. Eur J Trauma. 2004;4: Deleted in proof. 45. Johner R, Wruhs O. Classification of tibial shaft fractures and correlation with results after rigid internal fixation. Clin Orthop. 1983: Chapman CB, Herrera MF, Binenbaum G, et al. Classification of intertrochanteric fractures with computed tomography: a study of intraobserver and interobserver variability and prognostic value. Am J Orthop. 2003;32: Swiontkowski MF, Agel J, McAndrew MP, et al. Outcome validation of the AO/OTA fracture classification system. J Orthop Trauma. 2000;14: Bhandari M, Tornetta IP, Sprague S, et al. Predictors of reoperation following operative management of fractures of the tibial shaft. J Orthop Trauma. 2003;17: q 2005 Lippincott Williams & Wilkins 409