ORIGINAL INVESTIGATION

Transcription

1 ORIGINAL INVESTIGATION Do Subspecialists Working Outside of Their Specialty Provide Less Efficient and Lower-Quality Care to Hospitalized Patients Than Do Primary Care Physicians? Scott R. Weingarten, MD, MPH; Lynne Lloyd, MBA; Chiun-Fang Chiou, PhD; Glenn D. Braunstein, MD Background: Studies show that subspecialists can provide better quality care than primary care physicians when working within their subspecialty for patients with some medical conditions. However, many subspecialists care for patients outside of their chosen subspecialty. The present study compared the quality of care provided by subspecialists practicing outside of their specialty, general internists, and subspecialists practicing within their specialty. Methods: The severity-adjusted mortality rate and the severity-adjusted length of stay were used as indexes of quality of care. Data from 5112 hospital admissions (301 different physicians) for community-acquired pneumonia, acute myocardial infarction, congestive heart failure, or upper gastrointestinal hemorrhage at 6 hospitals in the greater Cleveland, Ohio, area were used in this study. The data were severity adjusted with the CHOICE Severity of Illness System. Results: Subspecialists working outside of their subspecialty cared for 25% of hospitalized patients. When comparing patients cared for by subspecialists practicing outside of their subspecialty, severity-adjusted lengths of stay were longer for patients with congestive heart failure (23% longer; 95% confidence interval [CI], 15%- 32%), upper gastrointestinal hemorrhage (22% longer; 95% CI, 7%-39%), and community-acquired pneumonia (14% longer; 95% CI, 5%-24%) than for patients cared for by subspecialists practicing within their subspecialty. Patients also had a slightly higher hospital mortality rate when cared for by subspecialists practicing outside of their specialty than by subspecialists practicing within their subspecialty (mortality rate odds ratio, 1.46; P=.047). In addition, patients cared for by subspecialists practicing outside of their subspecialty had longer lengths of stay, and prolongations of stay were observed for patients with congestive heart failure (16% longer; 95% CI, 8%-26%), upper gastrointestinal hemorrhage (15% longer; 95% CI, 2%-30%), and communityacquired pneumonia (18% longer; 95% CI, 9%-28%) than patients cared for by general internists. Conclusions: Subspecialists commonly care for patients outside of their subspecialty, despite the fact that their patients may have longer lengths of stay than those cared for by subspecialists practicing within their specialty or by general internists. In addition, such patients may have slightly higher mortality rates than those cared for by subspecialists practicing within their subspecialty. Arch Intern Med. 2002;162: From the Department of Health Services Research (Zynx Health, Inc), Cedars-Sinai Health System, Beverly Hills, Calif (Drs Weingarten and Chiou); and the Department of Medicine, University of California, Los Angeles, UCLA School of Medicine (Drs Weingarten and Braunstein). Ms Lloyd is an independent consultant. THERE HAS BEEN significant discussion and debate regarding the role of subspecialists and primary care physicians in providing care to patients with diverse medical conditions. 1-7 Several studies have reported that subspecialists have more up-to-date medical knowledge and provide better quality of care than primary care physicians when caring for patients with conditions within their chosen specialty (eg, cardiologists providing care to patients with congestive heart failure). 1 For example, when patients with acute myocardial infarction, acute nonhemorrhagic stroke, and asthma are cared for by subspecialists, they may have better outcomes than when they are cared for by general internists. 1 Moreover, a survey 8 of primary care physicians showed that primary care physicians believed that the scope of conditions that they treat had increased significantly, and that 24% believed that the scope of care that they were expected to provide was greater than it should be. Recent studies 9-11 have suggested that there may be a surplus of subspecialists, as determined by projecting physician manpower needs from managed care subspecialty requirements to a population of patients. A possible surplus of subspecialists may result in some subspecialists expanding the scope of care that they provide and treating conditions outside of their chosen specialty

2 SUBJECTS AND METHODS OUTCOME MEASURES The primary outcome measures used to indicate the quality of care that patients received were the severityadjusted mortality rate and hospital length of stay (LOS). The models were constructed based on patients demographic and clinical data. DATA SOURCE Six hospitals in the greater Cleveland area, in northeast Ohio, provided information on their physician subspecialties and patients cared for by these physicians to this study. All of these hospitals were members of the Cleveland Health Quality Choice Coalition Consortium Among them, 1 is a rural hospital and 5 are community hospitals. Of the 6 hospitals, only 1 had a hospitalist program, and none had an internal medicine training program, family practice training program, or full-time faculty. Two of the hospitals were part of a health care system and are coded as a single hospital (hospital 4) (Table 1). The Cleveland Health Quality Choice program was a regional effort of health care organizations to compare and improve hospital performance Lengths of stay and mortality rates for patients with acute myocardial infarction, congestive heart failure, upper gastrointestinal hemorrhage, or community-acquired pneumonia were examined in this study. Data, including sociodemographic variables, admission source, medications, medical history, vital signs, selected variables from the physical examination, results of laboratory tests, electrocardiographic findings, echocardiographic findings, and do not resuscitate status, were abstracted from the medical record of each patient by medical record technicians. There were explicit protocols for data abstraction, double keystroke entry, identification of outof-range variables, and independent verification of data quality at each hospital. PHYSICIANS AND PHYSICIAN CLASSIFICATION The primary physician for selected patients was obtained independently by each hospital participating in this study. The selectedphysicianwastheattendingphysicianofrecordineach case. The subspecialty status of physicians was verified by reviewing information supplied by each hospital (medical staff office), information provided on the American Medical Association and American Board of Internal Medicine Web sites, and other available information on physician subspecialty. 15 Physicianswereclassifiedasthosepracticingwithintheir subspecialty, thosepracticingoutsideoftheirsubspecialty, generalinternists, orfamilypractitioners. Forcommunity-acquired pneumonia, physicianswereclassifiedaspracticingwithinthe specialty if they were trained in infectious diseases or pulmonary medicine. For upper gastrointestinal hemorrhage, physicians were classified as practicing within the specialty if they were gastroenterologists. For congestive heart failure or acute myocardial infarction, physicians were classified as practicing within the subspecialty if they were cardiologists. PATIENTS AND PATIENT CLASSIFICATION Patients were classified based on the International Classification of Diseases, Ninth Revision, Clinical Modification, principal diagnosis code. Only information on patients who were hospitalized in the 6 hospitals for acute myocardial infarction, congestive heart failure, upper gastrointestinal hemorrhage, or community-acquired pneumonia between January 1, 1997, and December 1, 1997, was used in this study. Data of patients who were younger than 18 years or transferred from other acute-care hospitals were excluded. Patients may have been hospitalized more than one time, and each hospitalization was considered separately. SEVERITY ADJUSTMENT AND THE CHOICE SEVERITY OF ILLNESS SYSTEM The CHOICE Severity of Illness System was developed by the Cleveland Health Quality Choice program. 12,13 The Although many studies have compared subspecialists practicing within their chosen specialty with primary care physicians, few have examined the quality and efficiency of care provided by subspecialists practicing outside of their specialty. Using a valid severity of illness model, 12,13 the present study compared the quality of care provided by subspecialists caring for patients outside of their specialty with that provided by general internists and by subspecialists caring for patients within their specialty. RESULTS PATIENT DEMOGRAPHICS AND CLASSIFICATION There were a total of 6485 patient hospital admissions that were potentially eligible for the study. Of these patient hospital admissions, 1373 included patients who were not clearly identified as being primarily treated by an internal medicine subspecialist, a general internist, or a family practitioner. The remaining 5112 patient admissions were enrolled in the study. Among them, 1143 patients (22%) had an acute myocardial infarction, 610 (12%) had an upper gastrointestinal hemorrhage, 1946 (38%) had congestive heart failure, and 1413 (28%) had community-acquired pneumonia. When patients were classified according to the type of physician who provided their care, as seen in Table 2, a total of 1776 patients (35%) were treated by a physician practicing within his or her specialty, 1083 (21%) were treated by an internist without an identified specialty, 990 (19%) were treated by a family practitioner, and 1263 (25%) were treated by a subspecialist practicing outside of his or her specialty. There were 301 different physicians. The mean±sd age of the patients was 72.2±13.9 years, 93% were white, and 51% were men. About 72% of the patients had Medicare insurance, and 23% had commercial insurance. The mean±sd LOS was 5.6±3.9 days. IN-HOSPITAL MORTALITY The mean in-hospital mortality was 5.4%. The mean severity-adjusted mortality was 5.5%. Mortality rates for each hospital are listed in Table 1. Patients cared for by 528

3 system has multivariate models that were developed separately for each diagnosis. These models enable us to predict the mortality (0%-100%) and expected LOS (in days) for patients with community-acquired pneumonia, congestive heart failure, upper gastrointestinal hemorrhage, or acute myocardial infarction. These models were built from the demographic and clinical variables ascertained within the first 48 hours of hospitalization and were validated by the CHOICE Severity of Illness System in several steps. The initial models were derived from factors that independently contributed to the risk of death or LOS (P.01) in logistic or linear regression models. Length of stay data were log transformed because the data were heavily skewed. The LOS models excluded patients who died in the hospital or were transferred to other hospitals. The performance of the mortality models was examined by the receiver operating characteristic curve and calibration was examined by the Hosmer-Lemeshow goodnessof-fit test, 16 while the performance of the LOS models was assessed by the value of R 2 and by analysis of residuals. Models used in this study were those reestimated and examined using the larger data set. The receiver operating characteristic curve areas and R 2 were similar among the diagnoses studied. The receiver operating characteristic curve areas for mortality and the R 2 for LOS for the diagnoses are as follows: acute myocardial infarction, 0.89 and 0.19, respectively; congestive heart failure, 0.85 and 0.14, respectively; pneumonia, 0.88 and 0.25, respectively; and upper gastrointestinal hemorrhage, 0.91 and 0.23, respectively. Performance of the model was also similar in different types of hospitals. We used our data to calculate the area under the receiver operating characteristic curves, and found similar findings. STATISTICAL ANALYSIS Differences in patients demographics and outcomes between physician groups were analyzed: differences in patients age, LOS, and risk of death (severity-adjusted mortality rate) were examined using a Wilcoxon nonparametric test, while those in sex, race, and mortality were examined using a 2 test. Multivariate analyses were also performed to analyze differences in patients outcomes of physician groups and to adjust patient outcomes for patient severity of illness. Individual patient severity of illness was first determined based on each patient s demographics (eg, age) and clinical factors (eg, coexisting diseases, laboratory results, and vital signs) using the CHOICE Severity of Illness System, assuming no unmeasured selection effects associated with mortality rates and LOS due to the limitation of available data. To estimate the magnitude of the difference in severityadjusted LOS between the physician groups, a linear regression analysis was used. A dummy variable for each physician group was used within the model to ascertain the differences between each group relative to one another. Because LOS is log transformed, the antilog of the coefficient in the linear regression model represents the ratio of severity-adjusted LOS between each physician group and the reference group. If the coefficient is 0.1, for example, the ratio of severity-adjusted LOS between a specific physician group and the reference group is 1.26 (e 0.1 ). It can also be interpreted that the severity-adjusted LOS of the specific physician group is 26% higher than the one of the reference group. To analyze differences in mortality rates, logistic regression was used with a dummy variable to ascertain the differences in mortality rates of physician groups. With a certain formula of antilog transformation, the variable estimate for each dummy variable in the logistic regression model measures the change in the mortality rate between each physician group and the reference group. Confidence intervals were then calculated for each estimate to reflect the variation within the data and the statistical significance of the findings. P.05 was considered statistically significant. SAS statistical software was used for all of the statistical analyses. 17 subspecialists practicing outside of their specialty had higher mortality rates than those cared for by subspecialists practicing within their specialty (P=.047) (analysis 1 in Table 3). There were no significant differences in the mortality rates when comparing patients cared for by general internists with those cared for by subspecialists practicing outside of their specialty (P=.17) or when comparing patients cared for by general internists with those cared for by subspecialists practicing within their subspecialty (P=.65) (analysis 2 in Table 3). Similar results were found for the severity-adjusted mortality rate. Too few patients with an upper gastrointestinal hemorrhage died to compare mortality rates by physician types. LENGTH OF STAY The mean patient hospital LOS was 5.7 days. The mean LOS was 5.5 days for general internists patients, 5.6 days for family practitioners patients, 5.2 days for patients cared for by subspecialists practicing within their specialty, and 6.6 days for those cared for by subspecialists practicing outside of their specialty. The severity-adjusted LOS was longer for patients treated by subspecialists practicing outside of their specialty than for those cared for by subspecialists practicing within their specialty (Table 4). These differences were observed for patients with acute myocardial infarction, congestive heart failure, gastrointestinal hemorrhage, and pneumonia. In addition, patients cared for by subspecialists practicing outside of their subspecialty had longer LOSs than those treated by general internists (Table 4). COMMENT This study demonstrated that subspecialists caring for patients outside of their specialty may provide less efficient care, as evidenced by longer LOSs, than either subspecialists practicing within their subspecialty or general internists. In addition, patients cared for by physicians practicing outside of their specialty may have slightly higher mortality rates than those cared for by subspecialists practicing within their specialty. The odds ratio for subspecialists caring for patients outside of their subspecialty when compared with subspecialists caring for 529

4 Table 1. Observed Mortality Rates From Different Hospitals* All Diseases Combined Within-specialty subspecialists (n = 1776) General internist (n = 1083) Family practitioner (n = 990) Out-of-specialty subspecialist (n = 1263) No. of Missing Data Observed Mortality, % (n1/n2) Severity-Adjusted Mortality, % Mortality, % (n1/n2) Hospital 1 Hospital 2 Hospital 3 Hospital 4 Hospital (78/1549) (19/342) 6.42 (7/109) 6.14 (17/277) 5.02 (31/617) 1.96 (4/204) (52/1012) (13/186) 9.76 (4/41) 8.81 (14/159) 3.36 (17/506) 3.33 (4/120) (44/976) (15/290) 8.33 (12/144) 4.15 (8/193) 3.77 (8/212) 0.73 (1/137) (80/1225) (28/383) (11/63) 7.43 (13/175) 6.75 (17/252) 3.12 (11/352) *n1 denotes the number of patients who died in the hospital, and n2 denotes the total number of patients in each group. The number of patients in each hospital represents only those for whom we have information on mortality. Patients with no data on mortality were excluded in the calculation of mortality rate. Hospital 4 is actually 2 hospitals belonging to a single health system. P values for 2 tests for within-specialty, family practitioner, and out-of-specialty subspecialist physician types were.25,.04, and.01, respectively. Too few patients died in some of the hospitals to perform the 2 test for general internist. There were no significant differences in mortality rates between hospitals 1, 2, 3, and 4. Hospital 5 had a significantly lower severity-adjusted mortality rate than the other hospitals. Table 2. Demographics of Patients Treated by Different Types of Physicians* Within-Specialty Subspecialists (n = 1776) General Internists (n = 1083) Family Practitioners (n = 990) Out-of-Specialty Subspecialists (n = 1263) Age, mean ± SD, y 69.0 ± ± ± ± 13.7 Male 996 (56) 512 (47) 425 (43) 570 (45) White 1665 (94) 1001 (92) 929 (94) 1127 (89) Acute myocardial infarction 757 (43) 179 (17) 96 (10) 111 (9) Congestive heart failure 627 (35) 403 (37) 369 (37) 547 (43) Upper gastrointestinal hemorrhage 139 (8) 165 (15) 146 (15) 160 (13) Community-acquired pneumonia 253 (14) 336 (31) 379 (38) 445 (35) *Data are given as number (percentage) of patients except for age. Differences in demographics between patient populations between the 4 different physician types were significant at P.001. Table 3. Comparison of Patient Mortality Rates Among Physician Types* Analysis 1 Analysis 2 Observed Mortality, % OR (95% CI) P Value OR (95% CI) P Value Acute myocardial infarction Within-specialty subspecialist (n = 757) 8.36 Reference (0.47 to 2.62).81 General internist (n = 179) (0.38 to 2.11).13 Reference... Family practitioner (n = 96) (0.13 to 1.30) (0.12 to 1.73).25 Out-of-specialty subspecialist (n = 111) (0.54 to 2.79) (0.48 to 3.94).56 Congestive heart failure Within-specialty subspecialist (n = 627) 3.44 Reference (0.43 to 1.76).69 General internist (n = 403) (0.57 to 2.35).69 Reference... Family practitioner (n = 369) (0.38 to 1.89) (0.32 to 1.69).47 Out-of-specialty subspecialist (n = 547) (0.81 to 2.84) (0.67 to 2.57).43 Community-acquired pneumonia Within-specialty subspecialist (n = 253) 4.38 Reference (0.39 to 2.49).97 General internist (n = 336) (0.43 to 2.41).97 Reference... Family practitioner (n = 379) (0.58 to 2.93) (0.63 to 2.58).51 Out-of-specialty subspecialist (n = 445) (0.63 to 3.07) (0.69 to 2.70).37 All diseases combined Within-specialty subspecialist (n = 1776) 5.04 Reference (0.60 to 1.37).65 General internist (n = 1083) (0.73 to 1.67).65 Reference... Family practitioner (n = 990) (0.56 to 1.35) (0.49 to 1.26).32 Out-of-specialty subspecialist (n = 1263) (1.00 to 2.12) (0.88 to 1.98).17 *Analysis 1 performed using specialists practicing within their specialty as the reference group; analysis 2, using general internists as the reference group. OR indicates odds ratio; CI, confidence interval. 530

5 Table 4. Comparison of Severity-Adjusted Lengths of Stay Among Physician Types* Mean Observed LOS, d Regression Coefficient P Value % Difference in Severity-Adjusted LOS (95% CI) Analysis 1 Analysis 2 Acute myocardial infarction Within-specialty subspecialist (n = 757) 5.37 Reference... Reference 1.6 ( 8.2 to 12.5) General internist (n = 179) ( 11.1 to 8.0) Reference Family practitioner (n = 96) ( 9.8 to 14.0) 3.1 ( 10.4 to 18.6) Out-of-specialty subspecialist (n = 111) ( 1.5 to 25.2) 12.9 ( 2.1 to 30.1) Congestive heart failure Within-specialty subspecialist (n = 627) 3.94 Reference... Reference 5.5 ( 12.3 to 1.7) General internist (n = 403) ( 1.7 to 14.0) Reference Family practitioner (n = 369) ( 0.4 to 15.9) 1.4 ( 6.6 to 10.2) Out-of-specialty subspecialist (n = 547) (15.1 to 31.9) 16.4 (7.9 to 25.6) Upper gastrointestinal hemorrhage Within-specialty subspecialist (n = 139) 3.19 Reference... Reference 11.4 ( 22.1 to 0.7) General internist (n = 165) ( 0.8 to 28.2) Reference Family practitioner (n = 146) ( 1.0 to 28.5) 0.4 ( 12.3 to 13.1) Out-of-specialty subspecialist (n = 160) (7.3 to 39.0) 15.0 (1.5 to 30.3) Community-acquired pneumonia Within-specialty subspecialist (n = 253) 5.26 Reference... Reference 3.8 ( 5.3 to 13.7) General internist (n = 336) ( 11.4 to 6.3) Reference Family practitioner (n = 379) ( 9.4 to 8.1) 2.5 ( 5.6 to 11.4) Out-of-specialty subspecialist (n = 445) (4.5 to 24.1) 18.4 (9.3 to 28.3) All diseases combined Within-specialty subspecialist (n = 1776) 4.44 Reference... Reference 2.0 ( 6.4 to 2.6) General internist (n = 1083) ( 2.6 to 6.6) Reference Family practitioner (n = 990) ( 0.9 to 8.5) 1.6 ( 2.9 to 6.4) Out-of-specialty subspecialist (n = 1263) (13.9 to 24.1) 16.5 (11.6 to 21.7) *Analysis 1 performed using specialists practicing within their specialty as the reference group; analysis 2, using general internists as the reference group. LOS indicates length of stay; CI, confidence interval. patients within their subspecialty was 1.46 (P=.047). Patients cared for by physicians outside of their specialty also had 19% longer LOSs for the total population of patients, and significantly longer LOSs for patients with congestive heart failure, upper gastrointestinal hemorrhage, and community-acquired pneumonia. These differences suggest that subspecialists practicing outside of their specialty may provide less efficient care and possibly lower-quality care when compared with physicians providing care within their subspecialty. When comparing the mortality rates of patients treated by physicians practicing outside of their specialty with those of patients cared for by general internists, there were no statistically significant differences. However, LOSs for patients cared for by subspecialists practicing outside of their specialty were 17% longer than those of patients cared for by general internists, and prolongations of stay were observed for patients with congestive heart failure, upper gastrointestinal hemorrhage, and community-acquired pneumonia. Therefore, LOSs were shorter when patients were treated by general internists rather than subspecialists practicing outside of their specialty. This study is one of few that have examined the potential implications of having subspecialists care for patients outside of their subspecialty. The strengths of the present study include the following: (1) it had more than 5112 patients treated at 6 different hospitals and (2) a severity-of-illness adjustment was performed to minimize the chance that differences in LOSs and mortality rates could be attributed to differences in patient severity of illness. 12,13 This study also has limitations. First, teams of physicians often care for hospitalized patients, including different subspecialists. It can be difficult to attribute the efficiency or quality of care to a single physician or type of physician. However, the identified physician in the study was the primary attending physician of record as coded by the hospital. Although we attempted to control for the number of patients with a particular condition treated by a physician, we only had access to the information of patients treated in the hospitals that participated in this study. It is possible that a physician might have admitted patients to hospitals other than these 6. Therefore, the real volume of patients treated by physicians in this study might be higher than what was measured. Information regarding the volume of patients treated by different types of physicians could be inaccurate and, thus, was not used as a variable in the analyses. In addition, there were some differences in demographics between those patients cared for by general internists, subspecialists practicing within their subspecialty, and subspecialists practicing outside of their subspecialty. However, patients LOSs and mortality rates were adjusted for patient severity of illness, which should account for any difference that patient severity of illness or age might have had on LOS. In addition, the LOS may impact hospital mortality rates. Finally, this study used the patient as the unit of analysis rather than the hospital or the physician. 531

6 The observed differences in LOS may demonstrate that physicians caring for patients outside of their chosen specialty are less familiar with patients with these conditions because volume-outcome relationships have been shown for many conditions in medicine, and subspecialists practicing outside of their subspecialty may be a marker for low patient volume. Moreover, subspecialists frequently care for patients outside of their chosen subspecialty, because 25% of patients were cared for by subspecialists practicing outside of their subspecialty. A recent study 8 showed that many primary care physicians believe that the scope of conditions that they are expected to treat is greater than it ought to be. Because many subspecialists may perform primary care and treat hospitalized patients outside of their subspecialty, it is possible that subspecialists may have similar concerns that the scope of conditions that they treat outside of their subspecialty is greater than it should be. In conclusion, subspecialists commonly care for patients outside of their subspecialty. Patients cared for by subspecialists practicing outside of their subspecialty had longer LOSs and possibly higher mortality rates than those cared for by subspecialists practicing within their subspecialty; they also had longer LOSs when compared with those cared for by general internists. If patients are cared for by subspecialists practicing outside of their specialty, their LOSs, and possibly even mortality rates, may be higher than those of patients cared for by subspecialists practicing within their subspecialty. Accepted for publication July 16, We thank Dwain Harper, DO, for his assistance with this study. Corresponding author and reprints: Scott R. Weingarten, MD, MPH, Zynx Health, Inc, Cedars-Sinai Health System, 9100 Wilshire Blvd, Suite 655E, Beverly Hills, CA ( weingarten@zynx.com). REFERENCES 1. Harrold LR, Field TS, Gurwitz JH. Knowledge, patterns of care, and outcomes of care for general internists and specialists. J Gen Intern Med. 1999;14: Meyer GS, Jacoby I, Krakauer H, et al. Gastroenterology workforce modeling. JAMA. 1996;276: Rhee SO, Luke RD, Lyons TF, Payne BC. Domain of practice and the quality of physician performance. Med Care. 1981;19: Greenfield S, Nelson EC, Zubkoff M, et al. Variations in resource utilization among medical specialties and systems of care: results from the Medical Outcomes Study. JAMA. 1992;267: Kassirer JP. Access to specialty care. N Engl J Med. 1994;331: Ayanian JZ, Guadagnoli E, McNeil BJ, Cleary CD. Treatment and outcomes of acute myocardial infarction among patients of cardiologists and generalist physicians. Arch Intern Med. 1997;157: Jollis JG, DeLong ER, Peterson ED, et al. Outcome of acute myocardial infarction according to the specialty of the admitting physician. N Engl J Med. 1996; 335: St Peter RF, Reed MC, Kemper P, Blumenthal D. Changes in the scope of care provided by primary care physicians. N Engl J Med. 1999;341: Weiner JP. Forecasting the effects of health reform on US physician workforce requirement: evidence from HMO staffing patterns. JAMA. 1994;272: Cooper RA. Seeking a balanced physician workforce for the 21st century. JAMA. 1994;272: Demaria AN, Lee TH, Leon DF, et al. Effect of managed care on cardiovascular specialists: involvement, attitudes and practice adaptations. J Am Coll Cardiol. 1996;28: Rosenthal GE, Harper DL. Cleveland Health Quality Choice: a model for collaborative community-based outcomes assessment. Jt Comm J Qual Improv. 1994; 20: Rosenthal GE, Harper DL, Quinn LM, Cooper GS. Severity-adjusted mortality and LOS in teaching and nonteaching hospitals: results of a regional study. JAMA. 1997;278: Rosenthal GE, Quinn LM, Harper DL. Declines in hospital mortality associated with a regional initiative to measure hospital performance. Am J Med Qual. 1997; 12: Reed Elsevier Inc and the American Board of Medical Specialists. The Official ABMS Directory of Board Certified Medical Specialists. 30th ed. New Providence, NJ: Marquis Who s Who A Division of Reed Elsevier Inc; Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143: SAS Software: SAS/Base and SAS/Stat, Version Cary, NC: SAS Institute Inc;