Cytohistologic Discrepancies A Means to Improve Pathology Practice and Patient Outcomes

Transcription

1 Anatomic Pathology / CYTOHISTOLOGIC DISCREPANCIES Cytohistologic Discrepancies A Means to Improve Pathology Practice and Patient Outcomes Karen M. Clary, MD, Jan F. Silverman, MD, Yulin Liu, MD, PhD, Charles D. Sturgis, MD, Dana M. Grzybicki, MD, PhD, Laura K. Mahood, MS, SCT(ASCP), and Stephen S. Raab, MD Key Words: Quality assurance; Correlation; Patient safety; Error; Report card Abstract The use of cytohistologic discrepancies to investigate and reduce error seldom is studied. All gynecologic discrepancies (n = 283; 0.87% and 7.37% of all cytologic and histologic cases, respectively) and nongynecologic discrepancies (n = 146; 2.26% and 0.44% of all cytologic and histologic cases, respectively) for 26 months were classified as sampling or interpretive. Specimen type and pathologist discrepancy percentages, effect of discrepancies on patient outcome, and interobserver agreement of discrepancies were evaluated. Discrepancies were interpretive in 67% and 34% of gynecologic and nongynecologic cases, respectively. Statistically significant associations were seen between individual pathologist and discrepancy percentages. Breast (1.2%) and bronchial (0.8%) cytologic diagnoses had the highest discrepancy percentages. The kappa scores ranged from 0.02 to 0.45 for pairwise agreement of discrepant cases. Of nongynecologic interpretive discrepancies available for review, 63% (27/43) and 14% (6/43) were of no or minor clinical significance, respectively. Cytohistologic correlation is a useful tool to monitor performance and to identify specimen types prone to error. Monitoring cytohistologic discrepancies is a useful quality assurance tool in the cytology laboratory. 1 As part of a continuous quality improvement program, cytohistologic correlation may help laboratories refine diagnostic criteria and improve diagnostic accuracy and reproducibility. Cytohistologic correlation entails the concomitant review of cytologic and histologic specimens that were obtained within a narrow time frame from the same site in a given patient. The level of agreement between cytologic and histologic diagnoses may be used as a measure of laboratory quality. Although such monitoring is a common practice in most laboratories, and is a requirement of the Clinical Laboratory Improvement Amendments of 1988, 2 the use of a correlation program to reduce error and improve patient safety seldom has been studied. Most discussions of error in diagnostic pathology have concentrated on an individual case approach to error identification. 3 Anecdotal reports of diagnostic error are typically those that led to adverse patient outcomes. Many minor errors likely go unnoticed because of the lack of programs designed to systematically detect them. Few objective studies of error in pathology have been performed apart from interobserver variability studies and studies of false-positive and false-negative diagnoses. Interobserver variability studies carried out in certain areas of surgical pathology and cytopathology have shown substantial diagnostic variation, even among experts. 4,5 Using the metric of the cytohistologic discrepancy, several avenues of investigation were carried out to better elucidate the nature of the errors in cytology and histology, as well as to examine the effects on patient outcomes. The study of individual cytohistologic discrepancy rates also was Am J Clin Pathol 2002;117:

2 Clary et al / CYTOHISTOLOGIC DISCREPANCIES Table 1 Classification of Gynecologic and Nongynecologic Diagnoses Nongynecologic Cytology and Gynecologic Histology Gynecologic Cytology Histology Benign Within normal limits Benign Atypical Atypical squamous cells of undetermined significance Suspicious Low-grade squamous intraepithelial lesion CIN 1 Malignant High-grade squamous intraepithelial lesion CIN 2/3 Carcinoma Carcinoma CIN, cervical intraepithelial neoplasia. used to prepare report cards for pathologists. Physician report cards are standardized reports on quality of performance using various quantifiable measures such as patient outcomes data. 6 Materials and Methods A cytohistologic discrepancy was defined as a 2-step or greater difference between a cytologic and histologic diagnosis made on a patient within a 6-month period. In Table 1, the broad classification of gynecologic and nongynecologic diagnoses is shown. For nongynecologic discrepancies, the same classification system was used for both cytology and histology. An example of a 2-step difference would be a cytologic diagnosis of atypical and a histologic diagnosis of malignant. For gynecologic discrepancies, the Bethesda system was used for cytologic classification, 7 and the cervical intraepithelial neoplasia (CIN) system was used for histologic classification. 8 An example of a 2-step difference would be a cytologic diagnosis of atypical squamous cells of undetermined significance (ASCUS) and a histologic diagnosis of CIN 2/3. Cytologic diagnoses of ASCUS cannot exclude high-grade squamous intraepithelial lesion (SIL), and atypical glandular cells of undetermined significance were excluded from the analysis. Between January 1, 1998, and February 29, 2000, all cytohistologic gynecologic discrepancies (n = 283) and nongynecologic discrepancies (n = 146) were detected by computer retrieval of the Allegheny General Hospital (Pittsburgh, PA) pathology data files (Sunquest Information Systems, Flexilab version 5.2, Tucson, AZ). As part of the regular quality assurance program performed by one of us (S.S.R.), each cytohistologic discrepancy was reviewed in an unblinded fashion and classified as sampling error or interpretive error. Interpretive errors were further classified as undercalls or overcalls on cytology or histology. A disparity between the original diagnosis and the review diagnosis was considered an undercall when the original diagnosis was less severe than the review diagnosis (by 1 step or more; Table 1) and an overcall when the original diagnosis was more severe. The 4 subcomponents of the study were the assessment of (1) individual pathologists discrepancy rates, (2) specimen type discrepancy rates, (3) interobserver diagnostic variability, and (4) effect on patient outcomes. Individual Pathologists Discrepancy Rates Gynecologic and nongynecologic interpretive discrepancy rates were determined for 9 pathologists based on total numbers of cases signed out. Discrepancy rates were defined as the proportion of cases with interpretive cytologic or histologic errors during the period examined. The pathologists varied in levels of experience. Pathologists 2 through 7 and 9 had more than 5 years experience signing out cases, while pathologists 1 and 8 had fewer than 5 years experience. Pathologists 1 through 7 signed out both cytology and histology cases, while pathologists 8 and 9 signed out histology cases only. Specimen Type Discrepancy Rates Of the nongynecologic discrepancies, interpretive discrepancy rates were determined for specific cytologic and histologic specimen types. These were further subdivided into undercall and overcall discrepancy rates. Cytologic specimens were subclassified as bronchial (including bronchial brush and wash specimens), esophageal brushing, pleural fluid, peritoneal fluid, urine, and fine-needle aspiration (FNA) including breast, brain, and liver. Histologic specimens, including biopsies and resections, were subclassified as pulmonary, breast, esophagus, thyroid gland, and common bile duct. The denominators were total cases of each specimen type in cytology or histology during the 26- month study period. Specimen types with highest interpretive error rates were identified. Reasons for these errors were evaluated, including few malignant or atypical cells present, grading error (eg, reactive vs atypical vs malignant), or misinterpreted cells (eg, carcinoma misinterpreted as atypical squamous metaplasia). Interobserver Diagnostic Variability The unblinded reviewer s determination as to the cause of the discrepancy was used as the gold standard to determine whether a cytologic or a histologic error had occurred. For each nongynecologic, cytohistologic interpretive discrepancy, 3 pathologists (J.F.S., Y.L., C.D.S.) blinded to the original diagnosis reviewed the glass slide portion of the discrepant pair classified as the interpretive error (eg, cytology or histology, but not both). Forty-three of 49 cases were available for review. Each reviewed case was categorized as benign, atypical, suspicious, or malignant. 568 Am J Clin Pathol 2002;117:

3 Anatomic Pathology / ORIGINAL ARTICLE Effect on Patient Outcomes The effect of discrepancies on patient outcomes (additional procedure needed, delay in diagnosis, morbidity, or mortality) was measured by analysis of hospital databases. A chart review was performed for each patient with a nongynecologic discrepancy, and data recorded were the original cytologic and histologic diagnoses and the clinical follow-up. Follow-up ranged from 0 to 31 months (mean, 6.5 months). Four charts were unavailable for review. Based on these chart summaries, the same 3 pathologists graded the clinical significance of the nongynecologic interpretive errors using a scale ranging from 1 to 5 (low to high severity). The codes for the scale were arbitrarily assigned as follows: 1, diagnosis had no significance (no additional procedure required for diagnosis because the cytologic and histologic specimens were obtained concurrently and the diagnosis was accurately given by examination of one or the other specimen type); 2, diagnosis resulted in increased surveillance or biopsy procedures (eg, increased endoscopic surveillance for diagnosis of Barrett esophagus and lowgrade dysplasia); 3, diagnosis led to an additional invasive surgical procedure (eg, segmental breast resection) or delay in treatment of fewer than 6 months; 4, diagnosis led to unnecessary treatment (eg, chemotherapy or radiotherapy) or a definitive surgical procedure or delay in treatment of more than 6 months; 5, diagnosis led to loss of life. Mean severity scores for each nongynecologic discrepancy were calculated; mean scores of less than 1.5 were interpreted as not clinically significant. Data Analysis Initially, data were analyzed using descriptive statistics, including measurement of central tendency (mean). Statistically significant associations between individual pathologists and the proportions of cases with interpretive cytologic and histologic errors were examined using the chi-square test. Statistically significant associations between individual nongynecologic specimen types and the proportions of cases with interpretive cytologic and histologic errors were examined using the chi-square test. Pairwise kappa statistics were used to measure the level of interobserver agreement between the diagnoses of nongynecologic cases with interpretive errors made by the 3 blinded reviewers. Paired comparisons also were made between the diagnoses made by each of the 3 blinded reviewers and the unblinded reviewer and between the 3 blinded reviewers and the original diagnoses (obtained from the original pathology reports). Some experts have attached the following qualitative terms to kappa scores: 0.0 to 0.2, slight agreement ; 0.2 to 0.4, fair ; 0.4 to 0.6, moderate ; 0.6 to 0.8, substantial ; and 0.8 to 1.0, almost perfect. 9 The level of interobserver agreement among the 3 pathologists severity scores was determined using pairwise kappa statistics. Statistically significant differences in the group mean error severity score for the different nongynecologic specimen types was examined using a 1-way analysis of variance and post hoc t tests. For all statistical analyses, statistical significance was assumed at a P value of.05 or less. Results Overall Discrepancy Rates During the 26-month study period, there were 283 gynecologic cytohistologic discrepancies (0.87% of all cytologic cases; 7.37% of all histologic cases) and 146 nongynecologic discrepancies (2.26% of all cytologic cases; 0.44% of all histologic cases). Of the gynecologic discrepancies, 67% and 33% were due to interpretive and sampling errors, respectively. Of the gynecologic interpretive discrepancies, 73% were a result of histologic error and 27% a result of cytologic error. Of the gynecologic histologic interpretive errors, 88% were due to undercalls on biopsy interpretation. The majority of biopsies classified as undercalls (86%) were interpreted originally as negative and reclassified as low-grade SIL on review; 13% were interpreted originally as negative or low-grade SIL and were determined to be high-grade SIL on review. Of the nongynecologic discrepancies, 34% and 66% were due to interpretive and sampling errors, respectively. Of the nongynecologic interpretive errors, 68% were a result of cytologic error and 32% a result of histologic error. Of the nongynecologic cytologic interpretive errors, 85% were due to undercalls on cytologic interpretation. Individual Pathologists Discrepancy Rates For gynecologic specimens, the interpretive cytologic discrepancy rate per pathologist ranged from 0% to 1.3% (mean, 0.70%), and the interpretive histologic discrepancy rate ranged from 0% to 5.7% (mean, 3.45%) Table 2 and Table 3. A statistically significant association between individual pathologists and discrepancy rates was determined for gynecologic cytology specimens (P =.012) and for gynecologic histology specimens (P =.012). For nongynecologic specimens, the interpretive cytologic error rate per pathologist ranged from 0% to 2.5% (mean, 0.48%); a statistically significant association between individual pathologists and discrepancy rates was determined (P =.00). The interpretive histologic error rate ranged from 0% to 0.07% (mean, 0.05%); the association between individual pathologists and discrepancy rates was not statistically significant for nongynecologic histologic specimens (P =.736). Am J Clin Pathol 2002;117:

4 Clary et al / CYTOHISTOLOGIC DISCREPANCIES Table 2 Cytologic Interpretive Discrepancies by Pathologist for Specimens Signed Out * Nongynecologic Specimens Gynecologic Specimens Pathol- No. of No. of ogist Specimens Undercall Overcall Total Specimens Undercall Overcall Total 1 2,266 9 (0.40) 2 (0.09) 11 (0.49) 1,840 5 (0.27) 7 (0.38) 12 (0.65) 2 1,729 0 (0.00) 0 (0.00) 0 (0.00) 1,285 2 (0.16) 3 (0.23) 5 (0.39) 3 1,335 6 (0.45) 0 (0.00) 6 (0.45) (0.1) 7 (1.0) 8 (1.1) (0.9) 0 (0.00) 7 (0.9) (0.3) 5 (0.8) 7 (1.1) (0.6) 1 (0.2) 4 (0.8) (0.0) 1 (0.3) 1 (0.3) (1.3) 2 (1.3) 4 (2.5) (1.3) 0 (0.0) 2 (1.3) (1) 0 (0) 1 (1) 57 0 (0) 0 (0) 0 (0) Total 6, (0.41) 5 (0.09) 33 (0.48) 4, (0.24) 23 (0.46) 35 (0.70) * Data are given as number (percentage). Data for pathologists 8 and 9 are not included because they do not sign out cytology. The total number of gynecologic interpretive discrepancies includes 16 undercalls made by cytotechnologists that are not included in this table. Table 3 Histologic Interpretive Discrepancies by Pathologist * Nongynecologic Specimens Gynecologic Specimens Pathol- No. of No. of ogist Specimens Undercall Overcall Total Specimens Undercall Overcall Total (0.0) 0 (0.0) 0 (0.0) (0.0) 0 (0.0) 0 (0.0) 2 1,131 0 (0.00) 0 (0.00) 0 (0.00) (0.0) 0 (0.0) 0 (0.0) 3 2,819 0 (0.00) 0 (0.00) 0 (0.00) (1.8) 2 (0.5) 9 (2.3) 4 5,377 1 (0.02) 3 (0.06) 4 (0.07) (3.0) 2 (0.3) 21 (3.3) 5 3,779 1 (0.03) 1 (0.03) 2 (0.05) (1.7) 3 (0.7) 10 (2.4) 6 5,821 1 (0.02) 2 (0.03) 3 (0.05) (3.5) 2 (0.3) 22 (3.8) 7 4,456 3 (0.07) 0 (0) 3 (0.07) (4.1) 1 (0.2) 22 (4.2) 8 3,116 0 (0) 0 (0) 0 (0) (4.0) 2 (0.6) 16 (4.5) 9 4,046 1 (0.02) 1 (0.02) 2 (0.05) (5.1) 2 (0.5) 21 (5.7) Total 31,290 7 (0.02) 7 (0.02) 14 (0.04) 3, (3.05) 14 (0.40) 121 (3.45) * Data are given as number (percentage). Three pathologists left the practice during the study period and had low numbers of specimens; their data were excluded from the analysis of individual pathologists. However, their 18 gynecologic and 2 nongynecologic interpretive discrepancies were included in the total numbers of gynecologic (n = 283) and nongynecologic (n = 146) discrepancies. Specimen Type Discrepancy Rates For the most common specimens (>25 total cases per year), the highest discrepancy rate was for breast FNA specimens (1.2%), followed by bronchial cytology (0.80%) Table 4. Higher discrepancy rates were seen for some rare specimen types such as brain FNA specimens (5%) and common bile duct biopsy specimens (5%). A statistically significant association was seen between cytologic or histologic specimen types and discrepancy rates. Statistically, more errors than expected were seen for specimens of breast FNA and bronchial cytology, as well as brain FNA and common bile duct biopsy (P =.00). Statistically, fewer errors than expected were seen for specimens from breast biopsies and resections (0.04%; P =.00). Most cytologic errors were false-negative diagnoses (64% [18/28]), and these cases often consisted of well-differentiated neoplasms of scant cellularity. The 7 errors for breast FNA specimens included 5 undercalls and 2 overcalls on cytology. Of these 7 cytohistologic discrepancies, in only 2 cases (29%) were concurrent biopsy specimens obtained at the time of cytologic examination, while for the remainder, biopsy specimens were obtained nonconcurrently. This percentage is lower than the percentage of concurrent cytology-histology specimens compared with other specimen types such as bronchial (64% [7/11]) and esophageal cytology (100% [3/3]). Interobserver Diagnostic Variability There was moderate diagnostic agreement between the 3 blinded reviewers using pairwise comparisons, as measured by the kappa statistic Table 5. There was more variability (slight to fair agreement) between the 3 blinded 570 Am J Clin Pathol 2002;117:

5 Anatomic Pathology / ORIGINAL ARTICLE Table 4 Nongynecologic Interpretive Discrepancies by Specimen Type * Specimen Type Total No. of Cases Undercall Overcall Total Mean Severity Score Cytology Bronchial cytology 1, (0.72) 1 (0.07) 11 (0.80) 1.6 Esophageal cytology (0.6) 1 (0.3) 3 (0.8) 1.2 Pleural fluid (0.3) 0 (0) 2 (0.3) 1.0 Peritoneal fluid (0.5) 0 (0) 2 (0.5) 1.2 Urine (0.3) 0 (0) 1 (0.3) 1.7 FNA, breast (0.8) 2 (0.3) 7 (1.2) 2.3 FNA, brain 19 1 (5) 0 (0) 1 (5) 1.0 FNA, liver (0.7) 0 (0) 1 (0.7) 2.0 Histology Pulmonary 1,144 4 (0.35) 2 (0.17) 6 (0.52) 1.4 Breast 2,231 0 (0) 1 (0.04) 1 (0.04) 2.0 Esophagus 1,766 0 (0) 6 (0.34) 6 (0.34) 1.0 Thyroid gland (0.6) 0 (0) 1 (0.6) 1.7 Common bile duct 21 1 (5) 0 (0) 1 (5) 1.3 FNA, fine-needle aspiration biopsy. * Data are given as number (percentage). reviewers and the unblinded reviewer using pairwise comparisons. There was slight agreement between the 3 blinded reviewers and the original diagnoses. Effect on Patient Outcomes Of nongynecologic errors, 63% (27/43) were of no clinical significance (ie, score 1.5 or less and a concurrent biopsy or cytology was performed). A delay in treatment or an additional surgical procedure resulted from 37% (16/43) of all nongynecologic errors as reflected by mean severity scores of 1.5 or greater. Most commonly, these treatment delays and additional surgical procedures occurred in discrepant cases of breast FNA or biopsy specimens (75% [6/8] of discrepant breast cases) and pulmonary cytology or biopsy specimens (35% [6/17] of discrepant pulmonary cases). Only 6 (14%) of 43 nongynecologic errors were of moderate clinical import (severity score of 3 or greater). Of these, 5 resulted in treatment delays of 1 to 4 months, and 1 resulted in an unnecessary surgical procedure (segmental breast resection). Interobserver Variability in Severity of Outcomes Assessment There was fair to moderate agreement between the 3 pathologists ratings of severity of nongynecologic errors based on patient outcomes (ie, kappa = for reviewers 1 and 2, kappa = for reviewers 1 and 3, and kappa = for reviewers 2 and 3). There was a statistically significant association between error severity and specimen type, with a higher mean severity score for breast FNA specimens (2.3) compared with other specimen types (severity score of 2 or less) (Table 4). Table 5 kappa Statistics for Paired Comparisons of Diagnoses Original Unblinded Reviewer Diagnosis Reviewer Reviewer 2 Reviewer NA NA NA NA, not applicable. Discussion Cytohistologic correlation has been used widely as a quality assurance tool for cervicovaginal cytology and biopsy specimens The usefulness of nongynecologic correlation has been less well documented. This study demonstrates the use of cytohistologic correlation in both gynecologic and nongynecologic areas, with the emphasis on the latter. In contrast with most other correlation studies for gynecologic specimens, the present study showed the majority of discrepancies to be due to interpretive rather than to sampling error; most gynecologic discrepancies were attributed to histologic rather than to cytologic misinterpretation. This may be due to greater variation in thresholds for diagnosing cervical dysplasia in our laboratory. Before this study, little use was made of immediate cytohistologic correlation at the time of sign-out, and this lack of quality assurance may have led to more discrepancies. The majority of nongynecologic discrepancies were attributed to sampling error; of the interpretive errors, most were undercalls on cytology involving well-differentiated neoplasms and cases with few malignant cells. Am J Clin Pathol 2002;117:

6 Clary et al / CYTOHISTOLOGIC DISCREPANCIES Few studies have addressed monitoring individual pathologists performance as a quality assurance measure. 15,16 Monitoring cytohistologic discrepancy rates per pathologist serves to identify outliers who have more discrepancies and, perhaps, make more errors. In this series, statistically significant outliers were identified in gynecologic cytology and histology and in nongynecologic cytology. These results could lead to implementation of educational measures to improve interpretive skills. Report cards for physicians have been used or proposed in other branches of medicine to assess quality in patient care (eg, preventive care services, 17 cardiac surgery, 18,19 breast cancer care, 20 and prostate cancer care 21 ). Report cards are standardized reports on quality of performance that may be quantified and shared with users of the system for providing medical care. Under the influence of the managed care environment, physician report cards have been used increasingly. However, these efforts have not yet penetrated pathology practices. Several different methods of review are used in cytohistologic correlation studies, including review by 1 pathologist or by a panel of pathologists. 13,14,22 In the present study, 1 pathologist determined the nature of the errors by an unblinded review of the discrepant cases. This determination was arbitrarily deemed to be the gold standard. This could be viewed as a shortcoming owing to the subjectivity involved in cytology and biopsy interpretation. Yet, the alternative gold standard of histologic follow-up also is imperfect, as biopsy sampling problems are common, and histologic interpretive errors are just as frequent as cytologic errors. 11,23 One solution would be to have multiple reviewers validate the error assessment in the present study and use the consensus determination as the gold standard. As documented by the kappa statistics for the 3 blinded reviewers, significant interobserver variability was seen in the interpretation of cytologic and histologic specimens to which errors had been attributed. These kappa scores (0.435 to 0.452) are lower than kappa scores for the interpretation of more typical surgical pathology or cytopathology specimens (eg, tubular adenomas, ; low-grade SIL vs high-grade SIL cytology, ). The low kappa scores between pathologists indicate that a significant portion of these discrepant cases were interpreted differently by the reviewing pathologists. Thus, they may be considered difficult cases and may be prone to error. For individual specimen types, kappa scores may be used to identify those more prone to differences in opinion. As was done in the present study, these specimens may be reviewed to determine potential error sources. The 3 pathologists also disagreed with the original diagnoses in the majority of cases, indicating that true errors may have occurred. It is not surprising that the kappa statistics between the 3 blinded reviewers and the unblinded reviewer are low given the differences in the review processes. Little study of error according to different specimen types has been performed. In the present study, breast FNA was the specimen type with the highest interpretive discrepancy rate. Most studies of pathology error by specimen type focus on false-negative and false-positive rates and contributing factors for such errors. Studies on breast FNA specimens indicate more problems with underdiagnosis (more false-negative diagnoses [3.7%-6%] than falsepositive diagnoses [0%-0.68%]). 26,27 A similar trend was seen in the present study, although because we used a different kind of data set (cytohistologic discrepancy rather than review of all specimens for errors), additional comparisons may not be possible. Little discussion exists in the literature about how pathology discrepancies affect patient care. From these data, the effect on patient outcome is minor, although for some specimen types, the sample was small. Error significance scoring, as rated by the 3 reviewing pathologists, suggested that errors for breast FNA specimens had greater impact on patients than errors with other specimen types. Although one third of nongynecologic interpretive errors resulted in a delay in diagnosis, the effect on patient outcome seemed minimal. Error significance scoring rated only 14% (6/43) of interpretive errors as moderate severity (mean score, 3). No mean scores greater than 3 were seen, delays in treatment were confined to 6 months or fewer, and no definitive surgical procedures were carried out in error. No patients received adjuvant therapy (chemotherapy or radiation therapy) unnecessarily, and no deaths were caused by errors in diagnosis. Cytohistologic correlation is widely regarded as a useful quality assurance tool in cervicovaginal cytology. This study further demonstrates the usefulness of cytohistologic correlation in nongynecologic specimens. It may be used as a tool for evaluating individual pathologists performance and for identifying specimen types prone to more errors. In such a way, laboratories may develop educational protocols to improve interpretive skills and to identify problem areas with specific specimen types. From the Department of Pathology and Laboratory Medicine, Allegheny General Hospital, Pittsburgh, PA. Presented in part at the United States and Canadian Academy of Pathology annual meeting, Atlanta, GA, March Address reprint requests to Dr Clary: Dept of Pathology and Laboratory Medicine, Allegheny General Hospital, 320 E North Ave, Pittsburgh, PA Am J Clin Pathol 2002;117:

7 Anatomic Pathology / ORIGINAL ARTICLE References 1. Mody DR, Davey DD, Branca M, et al. Quality assurance and risk reduction guidelines. Acta Cytol. 2000;44: Medicare, Medicaid and CLIA programs: regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA), 57 Federal Register 7001 (1992). 3. Leape LL. Error in medicine. JAMA. 1994;272: Foucar E. Error identification: a surgical pathology dilemma. Am J Surg Pathol. 1998;22: Woodhouse SL, Stastny JF, Styer PE. Interobserver variability in subclassification of squamous intraepithelial lesions: results of the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology. Arch Pathol Lab Med. 1999;123: Hanchak NA. Managed care, accountability, and the physician. Med Clin North Am. 1996;80: Kurman RJ, Solomon D. The Bethesda System for Reporting Cervical/Vaginal Cytologic Diagnoses. New York, NY: Springer- Verlag; Crum CR, Nuovo GJ. Intraepithelial squamous lesions of the cervix. In: Crum CR, Nuovo GJ, eds. Genital Papillomaviruses and Related Neoplasms. New York, NY: Raven Press; 1991: Sackett DL, Haynes RB, Tugwell P, et al. The clinical examination. In: Sackett DL, Haynes RB, Tugwell P, et al, eds. Clinical Epidemiology: A Basic Science for Clinical Medicine. 2nd ed. Philadelphia, PA: Lippincott Williams & Wilkins; 1991: Sodhani P, Singh V, Das DK, et al. Cytohistological correlation as a measure of quality assurance of a cytology laboratory. Cytopathology. 1997;8: Jones BA, Novis DA. Cervical biopsy-cytology correlation: a College of American Pathologists Q-Probes study of 22,439 correlations in 348 laboratories. Arch Pathol Lab Med. 1996;120: Tritz DM, Weeks JA, Spires SE, et al. Etiologies for noncorrelating cervical cytologies and biopsies. Am J Clin Pathol. 1995;103: Joste NE, Crum CP, Cibas ES. Cytologic/histologic correlation for quality control in cervicovaginal cytology: experience with 1,582 paired cases. Am J Clin Pathol. 1995;103: Dodd LG, Sneige N, Villarreal Y, et al. Quality assurance study of simultaneously sampled, non-correlating cervical cytology and biopsies. Diagn Cytopathol. 1993;9: Houliston DC, Boyd CM, Nicholas DS, et al. Personal performance profiles: a useful adjunct to quality assurance in cervical cytology. Cytopathology. 1998;9: Wakely SL, Baxendine-Jones JA, Gallagher PJ, et al. Aberrant diagnoses by individual surgical pathologists. Am J Surg Pathol. 1998;22: Kim CS, Kristopaitis RJ, Stone E, et al. Physician education and report cards: do they make the grade? results from a randomized controlled trial. Am J Med. 1999;107: Hanlon CR. Quality assessment and tracking results of cardiac surgery. Ann Thorac Surg. 1997;64: Green J, Wintfeld N. Report cards on cardiac surgeons: assessing New York State s approach. N Engl J Med. 1995;332: West JG, Sutherland ML, Link JS, et al. A breast cancer care report card: an assessment of performance and pursuit of value. West J Med. 1997;166: Ullman M, Metzger CK, Kuzel T, et al. Performance measurement in prostate cancer: beyond report cards. Urology. 1996;47: Rohr LR. Quality assurance in gynecologic cytology: what is practical? Am J Clin Pathol. 1990;94: Stoler MH, Schiffman M. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL Triage Study. JAMA. 2001;285: Cross SS, Betmouni S, Burton JL, et al. What levels of agreement can be expected between histopathologists assigning cases to discrete nominal categories? a study of the diagnosis of hyperplastic and adenomatous colorectal polyps. Mod Pathol. 2000;13: Joste NE, Rushing L, Granados R, et al. Bethesda classification of cervicovaginal smears: reproducibility and viral correlates. Hum Pathol. 1996;27: Boerner S, Fornage BD, Singletary E, et al. Ultrasound-guided fine needle aspiration (FNA) of nonpalpable breast lesions: a review of 1885 cases using the National Cancer Institute supported recommendations on the uniform approach to breast FNA. Cancer. 1999;87: Westenend PJ, Sever AR, Beckman-deVolder HJC. A comparison of aspiration cytology and core needle biopsy in the evaluation of breast lesions. Cancer. 2001;93: Am J Clin Pathol 2002;117: