EVALUATION AND REFINEMENTS TO THE COMPREHENSIVE AMBULATORY CLASSIFICATION SYSTEM (CACS)

Transcription

1 CASEMIX, Volume 2, Number 4, 31 st December EVALUATION AND REFINEMENTS TO THE COMPREHENSIVE AMBULATORY CLASSIFICATION SYSTEM (CACS) Warren Skea, Daniel Benoit, Valérie Émond Canadian Institute for Health Information, Ottawa Communications to Warren Skea, Canadian Institute for Health Information, Ottawa 377 Dalhousie Street, Suite 200, Ottawa, Ontario K1N 9N8, Canada Tel , Fax , E mail casemix@cihi.ca ABSTRACT Background: The growth in ambulatory care and its associated costs, has created a need for grouping methodologies to be used for transforming data into information to predict resource requirements. As a result, a number of ambulatory care grouping methodologies have been developed over the past decade. The Canadian Institute for Health Information (CIHI) began the development of a National Ambulatory Care Reporting System (NACRS) in A part of this project was the creation of the Comprehens ive Ambulatory Classification System (CACS) with the aim of grouping NACRS cases from hospital based ambulatory care patients and community health centre cases into clinically relevant and resource homogeneous groups. The NACRS minimum data set upon which the CACS is based is made up of 40 data elements of which 17 are core required data elements. The grouping data elements are a sub-set of the NACRS minimum data set. The data elements are used to place ambulatory visits into mutually exclusive groups and include: main problem (diagnosis), main and other intervention(s), visit disposition, patient age, and gender. The final version of the CACS consists of 400 groups organized into 21 Major Ambulatory Clusters (MAC). Methods: The U.S. Army Ambulatory Care Database (ACDB) is a large (n=3.1 million records) cost database. CIHI submitted the CACS grouper to the U.S. Army Medical Information Systems and Services Agency who have previously evaluated all other major North American ambulatory care groupers. In keeping with the other evaluations, the grouper was tested based on three main criteria: 1) clinical usefulness, 2) administrative ease of implementation and 3) a statistical analysis of the grouper using cost as the dependent variable. A General Linear Model (GLM) was used to statistically assess the explanatory power of the groupers, as it may be the most appropriate type of test when dealing with unbalanced data (unequal number of cases in each group). Based on the statistical analysis and recommendations made by the investigative team, CIHI made further refinements to the CACS grouper by reducing the number of cells in the final version. In addition, a plan was formulated for further enhancement once more Canadian cost data is available. Results: The clinical evaluation found that the CACS was clear and appropriate and developed using sound logic. CACS was able to assign 96.5% of the visits contained in the data sample. Approximately 3.5% of visits were not grouped due to the decision to exclude based upon limited occurrences (< 30 observations) or ungroupable visits. The CACS grouper explained close to 29% of the variations in cost data (R 2 =0.288). This value surpassed all of the other groupers tested to date (next nearest value R 2 =0.219). Due to differences in coding systems (the U.S. utilizes the American Medical Association s Current Procedure Terminology: Canada utilizes the ICD-9) the obtained R 2 may be underestimating the explanatory power of the grouper. KEYWORDS: Case Mix, Methodology, Ambulatory care, Grouper Evaluation INTRODUCTION The growth in ambulatory care and its associated costs, has created a need for grouping methodologies to transform data into information to predict resource requirements. As a result, a number of ambulatory care grouping methodologies have been developed over the past decade. The Canadian Institute for Health Information (CIHI) completed the development of a National Ambulatory Care Reporting System

2 122 CASEMIX, Volume 2, Number 4, 31 st December 2000 (NACRS) in A part of this project was the creation of the Comprehensive Ambulatory Classification System (CACS) which is intended to group NACRS cases from hospital based ambulatory care and community health centres into clinically relevant and resource homogeneous groups. The NACRS minimum data set, upon which the CACS is based, consists of 40 data elements; 17 of which are core required data elements. The CACS grouping data elements are a sub-set of the core required elements. The data elements are used to place ambulatory visits into mutually exclusive groups and include: main problem (diagnosis), main intervention, visit disposition, patient age, and gender. Prior to the evaluation CACS consisted of 416 groups organized into 21 Major Ambulatory Clusters (MAC). The purpose of this article is to provide an overview of the analysis conducted to refine CACS resulting from an evaluation by the United States of America Army Medical Information Systems and Services Agency (U.S. Army, October 1998). CIHI chose the United States of America Army Medical Information Systems and Services Agency to evaluate the CACS grouper because it had been an independent evaluator of ambulatory care groupers for the past 15 years. As such, this would allow CIHI to compare the performance of the CACS to other ambulatory care groupers. This article does not attempt to explain the development of the CACS system rather is describes the process involved in the final post-evaluation revisions. It is important to understand what could not be completed when these refinements were done as well as the rationale of the decisions that were made prior to implementation. RESULTS OF THE U.S. ARMY EVALUATION The criteria designed to evaluate the CACS were similar to previous U.S. Army ambulatory care grouper evaluations. As such, the scope of the evaluation included: 1) clinical meaningfulness (i.e., from a clinical perspective did the groups make sense) 2) administrative ease of implementation; and 3) statistical analysis to test the predictive ability of costs by the grouper. The clinical evaluation indicated that the CACS was developed using sound medical logic although the clinical evaluation was not intended to be exhaustive. Moreover, CIHI has confidence around the clinical validity of the grouper since the physician responsible for this portion of the evaluation is the Chair of a University Medical Department, has extensive grouper experience, has military and civilian medical experience, and served as a member of the evaluation team (Cronson and Associates, 1996) that evaluated the Canadian Day Procedure Groups (DPG ). The criteria of administrative ease consists of two main components: 1) the installation and operation of the CACS software, and 2) the transparency of the grouping methodology, i.e., can one readily understand the logic of the grouping methodology. The grouper program is written in SAS language. Instructions on the installation and use appeared clear and appropriate. However, it was recommended that upon completion of the grouper algorithm development phase, the program should be re-written in a more efficient programming language (see recommendation section below). A series of statistical analyses (the General Linear Model, Duncan's Multiple Range Test, and Bonferoni and Scheffe t-tests) were conducted utilizing cost as the dependent variable. Results from these analyses indicated that the CACS explained nearly 29 percent of the variance in costs. It also illustrated that several groups or cells were similar in terms of resource consumption. The amount of variance in cost

3 CASEMIX, Volume 2, Number 4, 31 st December explained by the CACS grouper is currently the largest test statistic achieved by previous ambulatory care groupers* tested by the U.S. Army with the same data set. The U.S. Army evaluation made the following recommendations: 1) the grouping methodology should be written in a more efficient computer language, 2) a thorough review of all groups found to be similar in terms of resource intensity should be carried out and include both a resource and clinical component, and 3) the groups that did not meet a required statistical volume threshold of 30 cases should be reviewed with respect to administrative and clinical appropriateness. METHODS An Expert Panel was established with a number of CIHI case mix experts, as well as external experts in ambulatory care and grouper development. This Expert Panel reviewed the results of the U.S. Army analysis and provided the recommendations for these refinements. It was decided that the refinements would be completed in a systematic fashion so that other researchers could reproduce the steps if faced with a similar situation. This was extremely important since the review was conducted utilizing an unfamiliar data set from the U.S.A. Any decision to alter a group was based on clinical input and resource utilization (i.e resource homogeneity). The Expert Panel further agreed that the use of other ambulatory cost and/or activity data sets, if available, should be utilized to confirm or discard proposed changes. Following the examination of the information and data received from the U.S. Army, the Expert Panel decided that three analytic strategies would be employed to guide the grouper revisions. The strategies were: 1) a group dispersion analysis, 2) collapsing and splitting of CACS groups, and 3) a preliminary revision of the Day Procedure Groups (DPG). Data Preparation to Address U.S. Army Recommendations The refinements to CACS (cited above) were addressing recommendation number two (review of CACS groups for resource intensity) and three (review materiality of low volume CACS groups). As such, an empirical analysis of ambulatory care cost data was necessary to make the refinements to the CACS. The U.S. Army provided summary level cost data from the evaluation cost database that was used in the refinement process. In order to facilitate a more thorough analysis of CACS, the U.S. Army provided CIHI with a cost data file that included cost and frequency information per diagnosis for each CACS group. To perform this analysis it was necessary to prepare the data prior to the analysis. A total of 17,737 cases grouped into CACS cell 9999 Ungroupable. A frequency distribution of diagnosis codes was generated and reviewed. The majority of codes found in this group were invalid. There were also several cases with missing or extra digits. Changes made to ICD-9-CM between the time the data was collected ( ) and the year the diagnosis look-up table for CACS was created (1996) is probably the main reason for these discrepancies. Other ungroupable cases were due to special codes that have been developed and are used exclusively by the U.S. Army such as flight training physical. * Specifically, the study team had evaluated the Ambulatory Visit Groups (AVGs) formulated by a group of researchers from Yale University (Fetter, 1980), the Products of Ambulatory Care (PACs) developed by the New York State Health Department (Tenan et. al., 1988), the Emergency Department Groups (EDGs) Version 1.2, created by Health Systems Research, Inc. (Cameron, et. al., 1990), the Ambulatory Patient Groups (APGs) Version I (Averill, et. al., 1990) developed by 3M - Health Information Services (HIS), and Version II (Averill, et. al., 1995), the Ambulatory Care Groups (ACGs) designed by John Hopkins University (Weiner, Starfield, Steinwachs, and Mumford, 1990), and the Products of Ambulatory Surgery (PAS) created by the New York State Health Department (Filmore, et. al., 1991).

4 124 CASEMIX, Volume 2, Number 4, 31 st December 2000 ANALYSIS AND RESULTS The purpose of the analysis was to refine CACS on the basis of clinical relevance and increased resource homogeneity based on the cost information in the U.S. Army cost data file. The U.S. Army cost data file was used to develop the group dispersion analysis. That is, it allowed for the identification of groups where cases exhibited a wide range of costs. These groups could then be collapsed or split to improve their resource homogeneity, as long as the required clinical relevance was maintained. As such, clinical expertise was critical during this stage of the project. The CACS grouper is divided in 21 modules called Major Ambulatory Clusters (MAC). The following MAC were excluded from the group dispersion analysis: 1) Day Procedure Groups, 2) Mental Health, and 3) Rehabilitation. Day Procedure Group (DPG) The mapping between ICD-9-CM and Current Procedural Terminology (CPT-4) procedures for the DPG only includes a few commonalties. Forty of the 68 DPG groups had less than 30 observations, and 19 of those had no observation. The DPG grouper performs well with Canadian data. Work is currently underway to review the DPG and integrate it within the assigned MAC in CACS. Mental Health and Rehabilitation MAC The mapping between ICD-9-CM and CPT-4 procedures has few commonalties for mental health and rehabilitation. Eighty-four of the 87 mental health groups had no observation and one group had only 6 observations. In the rehabilitation area, 41 of the 65 groups had no observation and 17 had less than 30. Detailed group analysis should be performed on the Mental Health and Rehabilitation MAC once Canadian data with the corresponding intervention codes are available. STEP 1: Conduct a CACS Group Dispersion Analysis After excluding the DPG, Mental Health and Rehabilitation MAC, a total of 196 of the 416 CACS groups remained for the analysis. The purpose of the dispersion analysis was to examine the groups with the greatest variation in terms of cost per visit. During the development of the grouping methodology, three clinical areas were identified as having a high dispersion index (i.e. greater than 1.0 in the formula specified below in analysis 1). As such, special attention was paid to the following: Admitted Injury/Trauma Exam/Other The following analyses were conducted as part of Step 1: Dispersion Index Constraint, Volume Constraint, Grouper Activity Constraint, Attention focused on the Most Frequent Diagnoses within the CACS groups. Selected criteria and constraints helped identify groups with a large cost dispersion. Analysis 1: Dispersion Index Constraint Criteria: Dispersion indices greater than or equal to 1 in a CACS group. Dispersion Index Formula: Range of total cost for group (Max Min) Mean cost per group RESULT: 157 of 196 groups (80.1%) had a dispersion index greater than 1.0.

5 CASEMIX, Volume 2, Number 4, 31 st December Analysis 2. Volume Constraint (i.e. number of cases) Criteria: Volume within the CACS group greater than or equal to 30 with at least one diagnosis having at least 30 records. RESULT: 141 of the remaining 157 groups with a volume less than 30. Analysis 3. Grouper Activity Constraint Criteria: CACS groups that account for 80% of the total cost associated with the remaining 141 groups. Thirty-six CACS groups accounted for 80% of the total cost. These groups also accounted for 81% of the total case volume associated with the remaining 141 CACS groups. Due to the limited time and resources available, this step was taken in order to focus the investigation on groups that have a major impact on overall costs. Of the 36 CACS groups, the following four were excluded: Headache Other Unit Admission Nuclear Imaging CAT Scan By its nature, group 609 is widely dispersed and therefore it was agreed to exclude it from further analysis. Groups 2056 and 2057 are high intensity procedures. They are typically of low volume and should be examined using Canadian data to determine if the variation in cost is acceptable. Analysis 4. For the 32 remaining CACS groups, focus on the most frequent diagnoses within each of the CACS groups Criteria: Diagnosis codes with volume greater or equal to 100. Table 1 lists the 32 CACS groups examined. The Expert Panel examined data contained in the CACS groups found in Table 1. The data provided by the U.S. Army did not provide details on interventions provided during the visit, which made altering the groups difficult. The Panel decided to wait until either a complete Canadian ambulatory care cost data set, or the procedure information from the U.S. Army data becomes available, before changes are made to individual CACS groups. ISSUES Natural Clusters A nosological review of the 32 groups and of group 2002 Other Unit Admission is necessary. This action will take place when data including procedure information becomes available. The review will focus on the detailed diagnosis and intervention found in each group. It may be found that splitting a group based on natural clusters of diagnosis and procedures will introduce better clinical and resource homogeneity. Data Quality The issue of data quality arose throughout the analysis of the 32 groups. For example, in group 212 Vascular Problem the following codes for hypertension were present in the following distribution: Essential Hypertension Codes Malign 1 case Benign 2 cases Unspecified 10,000 cases

6 126 CASEMIX, Volume 2, Number 4, 31 st December 2000 Naturally, the vast majority of the volume and cost variation is found in the Unspecified diagnosis code. This type of issue is common in ambulatory care data sets. The group dispersion driven by the unspecified types of diagnosis is a limitation of coding in ambulatory care and can profoundly affect efforts to group such data. Table 1. CACS Groups meeting criteria for analysis CACS CACS Name Volume Dispersion 212 Vascular Problems Mgmt Diabetes, 18 + yrs Inv General ENT, 18 + yrs Mgmt General ENT, 0-17 yrs Mgmt General ENT, 18 + yrs Otitis Media Mgmt General Female Genital Disorders, yrs Mgmt General Female Genital Disorders, 45 + yrs Mgmt Contraception Mgmt General Male Genital Disorders, 18 + yrs Inv Other Genitourological Disorders, 18 + yrs Mgmt Other Genitourological Disorders, 18 + yrs Mgmt General Gastrointestinal, yrs Inv Other MSK & Conn Tissue, 18 + yrs Mgmt Inflam MSK & Conn Tissue, 18 + yrs Mgmt Other MSK & Conn Tissue, 18 + yrs Antepartum Routine Mgmt Oncology Mgmt Ophthalmology, yrs Mgmt Ophthalmology, 45 + yrs Inv General Respiratory, 18 + yrs Mgmt General Respiratory, yrs Mgmt Skin & Soft Tissue, 18 + yrs Mgmt Systemic Infection, yrs Closed Fracture & Dislocations Other Sprains Contusions except Fingers/Toes Mgmt General Symptoms/Exam Prophylactic Vaccination Therapeutic Counseling Routine Health Supervision Follow-up/Convalescence Issues specific to the U.S. Army Data Set The following example depicts an issue related to the use of specific Army codes in the database. There is no equivalency in the civilian community for Extensive Flight Physical Exams and therefore they are coded with the V-Code V70.0 Routine Medical Exam. The dispersion index of such codes will need to be examined again with Canadian data.

7 CASEMIX, Volume 2, Number 4, 31 st December Therapeutic Counseling (Group 2060) This group is predominantly populated with V-Codes containing a high degree of variability. It has been identified as having great dispersion since the development of the CACS grouping methodology. Again, the use of Canadian data with procedures will be useful for the revision of this group. Table 2. CACS Groups Collapsed based on U.S. Army Evaluation Previous Group # and Group Name New Group # and Group Name 201 Inv. General Circulatory, yrs 202 Inv. General Circulatory, yrs 201 Inv. General Circulatory, 0-17 yrs 203 Inv. General Circulatory, yrs 206 Mgmt General Circulatory, yrs 207 Mgmt General Circulatory, yrs 208 Mgmt General Circulatory, yrs 251 Inv Gen Endo/Nutr/Meta, yrs 252 Inv Gen Endo/Nutr/Meta, yrs 253 Inv Gen Endo/Nutr/Meta, 6-17 yrs 357 Inv Gen Male Genital Dis, 0-17 yrs 358 Inv Gen Male Genital Dis, 18+ yrs 452 Mgmt Hematology, 0-5 yrs 453 Mgmt Hematology, 6-11 yrs 454 Mgmt Hematology, yrs 551 Inv Infla MSK & Conn Tis, yrs 552 Inv Infla MSK & Conn Tis, yrs 553 Inv Infla MSK & Conn Tis, 6-11 yrs 554 Inv Infla MSK & Conn Tis, yrs 558 Mgt Infla MSK & Conn Tis, yrs 559 Mgt Infla MSK & Conn Tis, yrs 560 Mgt Infla MSK & Conn Tis, 6-11 yrs 561 Mgt Infla MSK & Conn Tis,12-17 yrs 751 Inv Ophthalmology, 0-11 yrs 752 Inv Ophthalmology, yrs 753 Inv Ophthalmology, yrs 755 Mgmt Ophthalmology, 0-11 yrs 756 Mgmt Ophthalmology, yrs 757 Mgmt Ophthalmology, yrs 206 Mgmt General Circulatory, 0-17 yrs 251 Inv Gen Endo/Nutr/Meta, 0-17 yrs 357 Inv Gen Male Genital Dis 452 Mgmt Hematology, 0-17 yrs 551 Inv Infla MSK & Conn Tis, 0-17 yrs 558 Mgt Infla MSK & Conn Tis,0-17yrs 751 Inv Ophthalmology, 0-44 yrs 755 Mgmt Ophthalmology, 0-44 yrs Future Refinement Options for Splitting the Identified 32 CACS Groups Three refinement options were identified by the Expert Panel and are discussed below: 1) Split by Categories. For example, Medical, Psychosocial, and other assuming they have similar mean costs. The advantages are these are simple, easy to do, and make intuitive sense. The disadvantages are a lot of work that may lead to minimal predictive power. 2) Develop three new intervention codes in conjunction with the three categories (Medical, Psychosocial, Other). This would account for time spent with the client (Brief, Intermediate, and Extensive). For example, Medical Brief. The advantages are that it provides a lot of information, resource homogeneity

8 128 CASEMIX, Volume 2, Number 4, 31 st December 2000 should be increased and intuitively appealing. However, the disadvantages are increased coding time and the introduction of new intervention codes. 3) Develop new diagnosis codes that would account for time spent with the client (Brief, Intermediate, Extensive). The advantages are the same as (2) above. There may be no need to collect two new data elements. The disadvantage is that any change to classification will mean that more new codes exist. STEP 2: Collapsing CACS groups An analysis examining CACS groups that could be collapsed based on low volume was conducted. A limited amount of 1996 Province of Alberta cost data was supplied by one of the Expert Panel members to validate the findings from the U.S. Army data. Table 2 shows the groups recommended to be collapsed by the Expert Panel. The Expert Panel recommended further examination of the following groups when more data is available: Mgmt Skin and Soft Tissue, 18 + yrs Closed Fracture & Dislocations Other Open Wound with Complications Minor Other Injuries Other Unit Admission EXPERT PANEL RECOMMENDATIONS Splitting CACS Groups The Expert Panel recommended waiting to split groups until data with procedure information is available. Review of Current DPG The Panel felt that the DPG module needed to be revised. As mentioned earlier, DPG is the grouping methodology used for day procedures in Canada. The grouping methodology uses principle procedure as the grouping variable to group cases in to one of the 69 DPG. This issue was consistent with the original plan. Preliminary work had been done and summary tables were reviewed. It was generally felt that the DPG needed to be revised in the near future since some groups may no longer be reflective of current same day surgery practices. In addition, new groups may need to be introduced in order to better reflect modern practice patterns. The DPG revision will include: the development of criteria to split DPG and create new groups; the possibility of using diagnoses as well as principle procedure to group; and the introduction of age splits It was suggested that further cost data would be required for the DPG revisions. An examination of the province of Alberta cost data identified the following DPG for future revision: DPG 2 Spinal Procedures DPG 4 Orbital and Other Eye Procedures DPG 5 Lens Procedures DPG 6 Iris and Other Eye Procedures DPG 8 External Eye Procedures DPG 14 Nasal Procedures DPG 16 External Ear Procedures DPG 28 Endoscopy GI

9 CASEMIX, Volume 2, Number 4, 31 st December These DPG were selected particularly because of their high dispersion index. Different researchers carried out a similar exercise using the Alberta provincial cost data. With the exception of DPG 28, the DPG listed here were also identified in Alberta for further investigation. It should be noted that the refinement to DPG that is required prior to migrating them into CACS is currently underway. Examine Recording / Coding guidelines The Expert Panel felt the need to look at the issue of recording guidelines in ambulatory care. Based on discussion, it was agreed that there is a need to form a committee with different skill sets to address issues regarding recording guidelines in ambulatory care. The committee should constitute experts in: Management Information Systems (MIS) Guidelines Coding/Classifications Grouping methodologies Health Care operational issues Education Specifically, recording guidelines in the area of Exam/Other should be revisited. A particularly contentious issue relates to the collection of visits. Visits occur in MIS functional cost centres in the CACS grouper. MIS guidelines currently provide organized and detailed standards for the collection, processing and reporting of management information related to staffing, costs, workload and provision of services. Through the implementation of the MIS guidelines, health care providers can develop management information systems that identify and track services and their accompanying costs. If a patient receives service from providers from differing functional centres then a different visit is recorded. The particular concern raised was that a patient may have multiple visits during one episode of care and that the resources to code these visits would be too cumbersome. However, the extent of this issue is currently unknown and an empirical investigation into this concern will be conducted once the collection of CACS data has occurred. CONCLUSION The first round of modif ications to the CACS grouper only included the collapsing of groups as per Table 2 of this document. As such, 16 CACS groups were deleted for a new total of 400 CACS groups. Future analysis recommended by the Expert Panel will include revising the grouper when comprehensive Canadian ambulatory cost data is available. The Expert Panel further recommended that a working group be formed to revise the recording guidelines in ambulatory care. Currently an extensive overhaul of the DPG is taking place as recommended by the Expert Panel as a precursor to their migration to CACS. At this time, the province of Ontario is pilot testing the NACRS data set to collect hospital based emergency department data. This data will then be grouped to CACS. It is anticipated that full mandatory collection of data using the NACRS data set in all ambulatory care settings in Ontario will be announced in the near future. The development of CACS weights will commence once comprehensive cost data is available. BIBLIOGRAPHY 1. Canadian Institute for Health Information (CIHI), (1999) National Ambulatory Care Reporting System. Project Report.

10 130 CASEMIX, Volume 2, Number 4, 31 st December Canadian Institute for Health Information (CIHI), (1999) Comprehensive Ambulatory Classification System: Summary Directory. 3. Canadian Institute for Health Information (CIHI), (1995) A Primer on Ambulatory Care Minimum Data Sets and Grouping Methodologies. 4. Moon, J, Georgoulakis, J.M. Bolling, D., Thristrup, L.J., Pierce, C.W. (1998) An Evaluation of the Canadian Institute for Health Information Comprehensive Ambulatory Classification System. U.S.A. Army Medical Information Systems and Service Agency.