DETERMINING AGREEMENT BETWEEN PHYSICIAN CLAIMS DATA AND MEDICAL CHART DOCUMENTATION FOR POLYPECTOMY

Transcription

1 DETERMINING AGREEMENT BETWEEN PHYSICIAN CLAIMS DATA AND MEDICAL CHART DOCUMENTATION FOR POLYPECTOMY Jonathan Michael Wyse Department of Epidemiology, Biostatistics and Occupational Health McGill University, Montreal Thesis submitted to the Faculty of Graduate Studies and Research in partial fulfillment of the requirements of the degree of Master of Science February Jonathan Michael Wyse i

2 ABSTRACT Background: Population level data on polypectomy rates may be useful in examining colorectal cancer (CRC) screening programs. Objectives: To determine level and predictors of agreement between physician claims database procedure code for polypectomy and polypectomy documentation in the endoscopy report. Methods: A retrospective cohort study of patients aged 50 to 80 years who underwent colonoscopy in Montreal was conducted. Physician claims records for the procedure code 0749 (polypectomy) were obtained from the Régie de l Assurance Maladie du Québec (RAMQ). Accuracy of the RAMQ database was assessed using the endoscopy report polypectomy recording as the gold standard. Results: Polypectomy procedure code in the RAMQ database had the following sensitivity: 84.7% [95% CI=79-89)], specificity: 99.0% [95% CI=98-100)], concordance: 95.1% [95% CI (93, 97)], and kappa statistic: 0.87 [95% CI (0.83, 0.91)]. No meaningful predictors of agreement were found. Conclusions: This study supports the use of physician claims databases as accurate sources of data in Quebec for identifying patients undergoing polypectomies. ii

3 RÉSUMÉ Mise en contexte: Des données sur les taux de polypectomie issues d études de populations peuvent être utiles dans l évaluation de programmes de dépistage de cancer colorectal. Objectifs: Déterminer les niveaux et facteurs prédictifs d accord entre les codes des actes pour polypectomie tels que documentés dans les bases de données de demandes de paiement de médecins et la documentation d une polypectomie dans le rapport endoscopique. Méthodes: Une étude de cohorte rétrospective de patients âgés entre 50 et 80 ans qui ont eu subi une endoscopie à Montréal fut entreprise. Les archives de demandes de paiement des médecins pour le code d acte 0749 (polypectomie endoscopique) fut obtenue de la Régie de l Assurance Maladie du Québec (RAMQ). La fiabilité de la base de données de la RAMQ fut évaluée en utilisant le rapport endoscopique comme talon de référence. Résultats: Les codes d acte de la RAMQ pour polypectomie eurent une sensibilité de 84.7% [95% CI (79, 89)], une spécificité de 99.0% [95% CI (98, 100)], une concordance de 95.1% [95% CI (93, 97)], et une statistique kappa de 0.87 [95% CI (0.83, 0.91)]. Aucun facteur prédictif utile ne fut identifié. Conclusions: Cette étude suggère que les bases de données de demandes de paiement de médecins sont des sources fiables de données Québécoises pour identifier les patients ayant eu une polypectomie endoscopique. iii

4 iv

5 STATEMENT OF SUPPORT While completing this research, I was supported by Dr. Maida Sewitch through a grant for New Investigators from the Fonds de la recherche en santé du Québec. v

6 ACKNOWLEDGMENTS I would like to thank those who contributed to the successful completion of my Master s thesis. Dr. Maida Sewitch, my supervisor, for her leadership and endless support throughout my Master s degree in Epidemiology, for providing countless hours of guidance and for herself being a truly motivating role model and researcher. Dr. Alan Barkun, my-co-supervisor, for being available, encouraging, and inspiring during my entire Gastroenterology fellowship and Master s degree training. Dr. Lawrence Joseph, a member of my thesis committee, for opening my mind to statistics with unparalleled enthusiasm, insight and dedication. Matthieu Storme for his assistance in chart reviews and data entry. Erin Arthurs and Dara Stein for their help with the database and communication with the RAMQ. The professors and administrative staff of the Department of Epidemiology, Biostatistics and Occupational Health at McGill University who paved the way for this wonderful journey. vi

7 DEDICATION I would like to dedicate this work to the countless patients I have encountered in my years thus far in medicine who continuously reaffirm my passion for research endeavors and clinical work. vii

8 TABLE OF CONTENTS ABSTRACT... ii RÉSUMÉ... iii STATEMENT OF SUPPORT... v ACKNOWLEDGEMENTS... vi DEDICATION... vii TABLE OF CONTENTS... viii LIST OF TABLES... x LIST OF FIGURES... xi LIST OF APPENDICES... xii LIST OF ABBREVIATIONS... xiii 1 RATIONALE LITERATURE REVIEW Preventing Colon Cancer Colonoscopy and Polypectomy Burden of CRC in Canada CRC Etiology: From Polyps to Cancer Detecting Polyps: CRC Screening and the Canadian Perspective Percentage of Colonoscopies Done for Screening Polyp Prevalence During Screening Colonoscopy Cecal Intubation Rates ( Successful or Complete Colonoscopies) Medical Claims Database Studies Overview of Medical Claims Database Studies Prescription Claims Database Studies Physician Claims Database Studies Database Research in the Gastroenterology Literature OBJECTIVES and HYPOTHESES Main Objective Secondary Objectives METHODOLOGY Study Design Overview Study Sites Study Populations Endoscopists Endoscopist Questionnaire Patient Questionnaire Endoscopy Report Physician Claims Records Patient Questionnaire Endoscopist Questionnaire Endoscopy Report Physician Claims Database Statistical Analysis Main Objective: Validity of Polypectomy Procedure Code viii

9 4.5.2 Objective #2: Predictors of Level of Agreement Objective #3: Comparison of our Study with the Literature Objective #4: Annual Procedure Frequency and Costs Confounders Ethical Considerations RESULTS Recruitment And Eligibility Descriptive Statistics Colonoscopy Procedure Code In The Ramq Database Main Objective: Validity Of RAMQ Polypectomy Procedure Code Objective #2: Predictors Of Level Of Agreement Objective #3: Comparison Of Our Study Findings With The Literature Percentage of Colonoscopies Done for Screening Purposes Percentage of Screening Colonoscopies with Polypectomy Objective #4: Annual Procedure Frequency And Costs DISCUSSION Colonoscopy Procedure Code In The Ramq Database Main Objective: Validity Of Polypectomy Procedure Code Objective #2: Predictors Of Level Of Agreement Objective #3: Comparison Of Our Study With The Literature Percentage of Colonoscopies Done for Screening Purposes Percentage of Screening Colonoscopies with Polypectomy Objective #4: Annual Procedure Frequency And Costs Limitatons Selection Bias Misclassification Bias Confounders Strengths CONCLUSIONS REFERENCES ix

10 LIST OF TABLES Table 1: Percentage Colonoscopies for Screening...7 Table 2: Highest Grade Lesion per Patient Undergoing Screening Colonoscopy...9 Table 3: Contingency Table for Level of Agreement...21 Table 4: Precision Estimates for Fixed Sample Size...22 Table 5: Patient Characteristics / Endoscopist Characteristics...26 Table 6: Contingency Table for Level of Agreement - Results...28 Table 7: Level of Agreement by Institution...29 Table 8: Level of Agreement by Specialty and Indication...29 Table 9: Level of Agreement by Endoscopist...30 Table 10: Number and Percent of Screening Colonoscopies by Endoscopist...31 Table 11: Literature: Screening Colonoscopies with Polypectomy...32 Table 12: Study Data: Screening Colonoscopies with Polypectomy...32 x

11 LIST OF FIGURES Figure 1: Colonoscopy 3 Figure 2: Polypectomy 4 Figure 3: Medical Administrative Database Studies Figure 4: Model of Main Objective and Objective # xi

12 LIST OF APPENDICES APPENDIX 1: BAYESIAN HIERARCHICAL MODEL 49 xii

13 LIST OF ABBREVIATIONS AGA BSG CAG COPD CRC CT CTC FOBT GIB IBD ICD-9 INR RA RAMQ American Gastroenterology Association British Society of Gastroenterology Canadian Association of Gastroenterology chronic obstructive pulmonary disease colorectal cancer computerized tomography computed tomographic colonography fecal occult blood testing gastrointestinal bleed inflammatory bowel disease International Classification of Diseases, ninth revision international normalized ratio research assistant Régie de l Assurance Maladie du Québec xiii

14 1 RATIONALE Colorectal cancer (CRC) is a major source of morbidity and mortality for Canadians. CRC screening across Canada relies heavily on the performance of colonoscopy, a procedure that allows direct visualization of the entire colon and rectum and permits immediate endoscopic removal of adenomatous polyps (referred to as polypectomy), the precursors of CRC. Physician claims databases are a frequently used source of population level data. In Quebec, physicians are reimbursed for medical services by the Régie de l Assurance Maladie du Québec (RAMQ), the government agency responsible for administering the provincial health insurance plan. To be reimbursed for a medical service performed, physicians must submit a claim to the RAMQ documenting the patient Medicare number, the date and location of service, the procedure code for the service provided (act performed), and one International Classification of Diseases, ninth revision (ICD-9) diagnostic code for the visit. Polypectomy is coded as a procedure separate from the colonoscopy itself and involves additional remuneration to the physician. There is only one code for a colorectal polypectomy during colonoscopy in Quebec, To our knowledge, there are no studies examining the number of polypectomies in Quebec relative to the number of colonoscopies done for screening purposes. Information on polypectomies would be valuable in assessing the quality of CRC screening. In 2007, complete colonoscopies performed in Quebec were the 3 rd most costly diagnostic or therapeutic procedure and the 11 th most frequently performed diagnostic or therapeutic procedure ( colonoscopies) [1]. Colonoscopies and polypectomies are procedures with significant financial implications throughout Canada, notably in cities such as Calgary, which opened a colonoscopy screening centre in early 2008 that expects to accommodate patients per year. The purpose of this study is to determine the utility of the RAMQ database in providing accurate information about polypectomy performance in Quebec, by examining the level and predictors of agreement between the polypectomy procedure code in the RAMQ database and the endoscopy 1

15 report in the patient medical chart. The RAMQ database will also be used to estimate the annual frequency and cost of patients undergoing a polypectomy during a screening colonoscopy by integrating our study results with the annual number of colonoscopies performed as recorded by the RAMQ. There are at least five advantages to the proposed research. First, to our knowledge, this will be the first study in Canada to assess the validity of procedure codes in gastroenterology as submitted in provincial physician claims databases compared to the patient medical chart, and the first study anywhere to examine polypectomies. Second, by studying the predictors of agreement between the two data sources, we will further define and characterize the database accurately for future studies. Third, this study may provide insight into the results of previous similar studies that did not validate procedure codes. Fourth, we will provide the first estimates of the frequency and costs of screening colonoscopies and resultant polypectomies in Quebec over a year period. Finally, studying polypectomy rates in Quebec may permit future critical evaluation of CRC screening and enable appropriate recommendations to health resource utilization on a provincial level. 2

16 2 LITERATURE REVIEW 2.1 PREVENTING COLON CANCER Colonoscopy and Polypectomy Colonoscopy is a procedure performed to visualize the distal portion of the gastrointestinal tract, in particular, the colon or large intestine. A flexible tube called a colonoscope is introduced into the anal canal and passed through the entire length of the colon until its proximal portion, the cecum. A fiberoptic or, more commonly today, video colonoscope permits inspection of the colon on a screen in real-time (Figure 1). The individual performing the colonoscopy is the endoscopist. Suspicious growths called polyps can be removed using a wire snare and electrocautery (Figure 2). Polypectomy is the term for polyp removal. Figure 1 Colonoscopy: a) video colonoscope b) colonoscope being passed through colon [2]. 3

17 Figure 2 Polypectomy: using a wire snare [3] Burden of CRC in Canada In Canada, CRC is currently the third most commonly diagnosed cancer in men and women. CRC is the second leading cause of cancer deaths in men and the third leading cause in women. In 2009, an estimated 22,000 new cases of CRC were diagnosed (male 12,100, female 9,900) and an estimated 9,100 deaths occurred (male 4,900, female 4,200) [4]. The 2009 estimated agestandardized incidence of CRC in Canada is 62 per 100,000 men and 41 per 100,000 women, while the age standardized mortality due to CRC is 26 per 100,000 men and 16 per 100,000 women. The lifetime risk of a male developing CRC is 7.4% (1 in 13.5) and dying from CRC is 3.7% (1 in 27.1). The lifetime risk of a female developing CRC is 6.5% (1 in 15.4) and dying from CRC is 3.3% (1 in 30.7) [4] CRC Etiology: From Polyps to Cancer Colonic polyps can be divided into those that are non-neoplastic and those that have the potential to become carcinomas and metastasize, referred to as adenomas. Based on significant epidemiologic, clinical, pathologic, and molecular studies, leading authorities in the field, agree that most, if not all, CRCs arise from adenomas that have acquired an increasingly dysplastic and/or malignant cellular phenotype [5]. 4

18 The exact progression time from adenoma to carcinoma is not entirely known given that polyp removal interrupts natural history and most studies have been retrospective. Polyp size greater than 1 centimeter, type of histology (villous versus tubular) and degree of dysplasia, increase the likelihood of becoming malignant. Most adenomatous polyps in most patients require 5-15 years for cancer to develop, ample time for detection and removal these lesions [5]. One challenge facing gastroenterology today is that endoscopists are unable to accurately predict within a given patient which adenomatous polyps are likely to progress to cancer and which are destined to remain non-detrimental in the long-term. Many polyps that are detected during a screening colonoscopy, removed, and found to be neoplastic may never have progressed to a truly malignant state. This dilemma is faced in other areas of oncology as well, such as the detection and treatment of prostate cancers that may in fact have remained indolent for decades [6]. Furthermore, at a macroscopic level, although polyps greater than 1 centimeter are almost all adenomas, even an experienced endoscopist cannot discern benign versus adenomatous histology at the time of colonoscopy. Thus, any polyp found on colonoscopy should be removed or at least biopsied [7] Detecting Polyps: CRC Screening and the Canadian Perspective Screening is defined as the examination of asymptomatic individuals in order to classify them as likely or unlikely to have the disease that is the object of the screening test [8]. CRC is considered an ideal disease for mass screening in those 50 years of age and older as it is a major cause of morbidity and mortality, has a known premalignant lesion and lengthy premalignant stage [9], occurs most often in an identifiable population (persons aged 50 and over), can be prevented by removal of adenomas, and cured by treatment of early-stage cancers [10]. The Canadian Task Force on Preventive Health Care Population has recommended screening for colorectal cancer since 2001 [11]. The Canadian Association of Gastroenterology (CAG) along with numerous international medical agencies [12-15] have not only put forth recommendations for CRC screening but also strongly support the establishment of population-wide screening programs for CRC. The most recently published CAG guidelines in 2004 are very similar to those previously published by the American Gastroenterology Association (AGA) and the 5

19 British Society of Gastroenterology (BSG). CAG suggested that the screening method used should be determined by its availability and discussion with the patient. Options include: 1) Fecal occult blood testing (FOBT) every 1 or 2 years, 2) Flexible sigmoidoscopy (with or without FOBT) every 5 years, 3) Double-contrast barium enema every 5 years, and 4) Colonoscopy every 10 years. Only a colonoscopy however, permits the simultaneous removal of polyps throughout the colon; the 10 year interval was allotted based on its superior sensitivity and specificity relative to the other tests [13]. However, the AGA recently updated their recommendations in March of 2008 with some significant differences compared with previously mentioned guidelines. Most notably, the AGA added a radiologic study using computerized tomography scanning (CT scanning) where insufflation of air into the colon is performed and images are then acquired. This is referred to as computed tomographic colonography (CTC) and is recommended as an option to be repeated every 5 years [14]. The AGA also newly divided testing into two categories: 1) tests on stool like FOBT that will primarily detect CRC itself, and 2) structural tests such as colonoscopy and CTC which detect both CRC and premalignant adenomatous polyps. A Quebec Association of Gastroenterology paper notes that in theory, using cost-effectiveness models, CRC screening is not expected to be cost-saving but that its implementation appears favorable when compared to breast and cervical cancer screening programs [16] Percentage of Colonoscopies Done for Screening An endoscopist may perform colonoscopies for a variety of indications such as diarrhea, rectal bleeding, surveillance following colon cancer resection and many others. In order to determine the proportion of colonoscopies performed for screening, a literature review was performed. The PubMed database was searched for the terms colonoscopy, polyp, polypectomy, screening, adenoma, and colorectal cancer. Studies were selected if they satisfied at least the first 3 criteria: 1) Studies involving endoscopy centers where all colonoscopies performed were included. Note: Some studies excluded some indications for colonoscopy. 2) The percentage of screening colonoscopies was stated. 3) Published in the 3 years prior to the present study: , in English. 4) Data collected in the 3 years prior to our study s data collection:

20 Three years was selected as the time frame because it was expected that the proportion of screening colonoscopies would have increased significantly over recent years given the global acceptance and publicity of screening colonoscopies, both in the scientific and secular communities. Our study was performed when both AGA guidelines [17] (published in 2003) and CAG guidelines [13] (published in 2004) were widely accepted. Table 1 lists the 10 studies identified from this search that met our first 3 criteria. They originated from all over the world, including countries in North America, Europe and Asia. All studies were published during the desired time period Only 6 of the 10 studies met all 4 criteria and had data collected during the desired years , whereas 4 studies used data collected from (Table 1 - highlighted in grey). The 6 studies with data collected between originated from the U.S. (2 studies) and 1 from each of England, Germany, Japan and Korea [18-23] and had a proportion of screening colonoscopies that ranged from 36.2 to 74.4%. Of the 6 identified studies meeting all criteria [18-23], three had relevant exclusion criteria [19, 20, 23] that may artificially increase the proportion of screening colonoscopies by excluding patients with known polyps or inflammatory bowel disease (IBD) who underwent colonoscopy (Table 1). Table 1: Percentage Colonoscopies for Screening* Study Location Year of Data Collection Number of Colonoscopies Number of Screening (%) Exclusion Criteria Rogart et al. [19] U.S (74.4) Polyposis syndromes, known adenoma, acute GIB, higher bleeding risk [international normalized ratio (INR) >2, platelets < 50,000] Lee et al. [23] Korea (42.9) Emergency procedure, GI obstruction, history of colon operations, therapeutic procedure, surveillance of inflammatory 7

21 Horiuchi et al. [21] bowel disease, age> 80 years. Japan (42.9) Pregnancy, ASA class III and IV, body weight >100 kg, East et al. [20] England (36.2) Age other than years, colitis, polyposis syndrome, major musculoskeletal problem. Wells et U.S (50.5) None al.[18] Adler et Germany Late 04, (56.7) None al. [22] Barclay et U.S. Jan (26.0) None al. [24] March 04 Leng et U.S (43) Population estimates al. [25] Population estimates Chen et U.S (27.3) None al. [26] Burnand et al. [27] Europe + Canada (range: ) None * Studies highlighted in grey did not meet the 4th criterion Polyp Prevalence During Screening Colonoscopy In order to determine the expected proportion of screening colonoscopies where a polypectomy is performed, the PubMed database was searched for the terms colonoscopy, polyp, polypectomy, screening, adenoma, and histology. Studies were selected if they satisfied the following criteria: 1) Screening colonoscopy in the general population, age 50 to 80 years of age. 2) Number of patients with at least one polyp was provided. 3) The most advanced histology per patient was provided. Therefore, the present literature review examined studies where outcome was the most advanced polyp per patient, which will provide an estimate of the number of patients who undergo 8

22 polypectomy during screening colonoscopy, as well as the distribution of the histology of the polyps. Four large studies satisfied the 3 criteria [28-31] (Table 2). For each study, the age range of patients is listed. The next 4 columns contain information on the most advanced polyp (or no polyp) found in each patient. For example, in Marbet et al. [30] 1912 patients underwent colonoscopy. Of those, 1404 did not have a polyp found. The remaining patients had polyps found ranging from hyperplastic (benign) to those containing cancer. The most advanced lesion per patient is listed. Table 2: Highest Grade Lesion per Patient Undergoing Screening Colonoscopy* Study Age No polyp Hyperplastic Adenoma High Risk Cancer Total range (%) (%) (%) Polyp (not (%) Colonoscopies cancer)** (%) Lieberman et al. [28] Strul et al. [29] Marbet et al. [30] Regula et al. [31] (50) 391 (13) 842 (27) 299 (10) 30 (1) (70) 69 (8) 145 (16) 50 (5) 11 (1) (73) 134 (7) 212 (11) 151 (8) 11 (1) (76) 3778 (9) 4180 (10) 2168 (5) 385 (1) Total * No polyp includes 1) absence of any polyp and a separate category included in two of the studies [28, 31] which were lesions or areas sampled not ultimately polypoid in nature. ** High Risk = villous histology, 1cm size, or high grade dysplasia, or any combination Cecal Intubation Rates ( Successful or Complete Colonoscopies) It is well accepted that cecal intubation rates (complete colonoscopies) should be very high during colonoscopy, and certainly during screening colonoscopy. The success rate for screening should be higher than non-screening in theory because the patient population is generally younger and healthier. A younger, cooperative patient without previous abdominal or pelvic 9

23 surgeries greatly facilitates the performance of colonoscopy. The reason for necessitating high completion rates is that incomplete colonoscopies do not visualize the entire colon; as a result, the patient is required to undergo additional testing that incurs further time, risks, and costs in order to ensure there were no missed polyps, or the patient does not undergo additional testing and the missed polyps may progress to cancer. A recent paper on Ontario colonoscopy standards found 14 studies (10 prospective) that detailed cecal intubation rates. The calculated weighted mean for success was 92%. Recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer in 2002 put forth guidelines of cecal intubation rates in all colonoscopies ( 90%) and in screening cases ( 95%). 2.2 MEDICAL CLAIMS DATABASE STUDIES Overview of Medical Claims Database Studies Studies that rely on administrative databases are ubiquitous in the medical literature. One way of summarizing medical claims database studies, is diagramed in Figure 3. Figure 3 - Medical Administrative Database Studies Medical claims databases are commonly used to conduct research [9, 32-42] and can be divided into two main types: 1) prescription claims databases and 2) physician claims databases. A brief 10

24 review of both types is provided in the following paragraphs using specific examples in each case. The majority of these studies do not attempt to validate the data used. A subset of studies that incorporate validation of the data will be highlighted. In these studies, the patient medical chart is often used as a gold standard to which the database is compared Prescription Claims Database Studies Prescription claims databases are used for phamacoepidemiological research, [33, 36, 41] and reflect the frequency and patterns of prescribing medications. One study in Quebec compared RAMQ prescription data with patients medical chart data, and showed a level of agreement of 83% [41] Physician Claims Database Studies Physician claims databases are commonly used in health services research [32, 34, 35, 37, 39, 42, 43] and have the important feature that they reflect physician remuneration. A physician making a claim to the agency responsible for administering the health insurance plan will need to include a code for the diagnosis (commonly the ICD-9 system), and if applicable, a procedure code for the service provided. Therefore, studies that rely on physician claims databases can be further divided into two major groups: 1) those that examine the diagnostic code entered in the claims and 2) those that examine the procedure code entered in the claims Studies Using the Diagnostic Code There are many examples of studies that rely on physician submitted diagnostic codes [32, 37, 39, 43], some of which have examined the validity of the data, as illustrated by the following 5 Canadian studies: Wilchesky et al. compared diagnostic codes in physician billing claims with the medical chart in order to examine the co-morbidities that comprise the Charlson Comorbidity Index. The authors found the specificity for most diagnostic codes was high (at least 90%), but the sensitivities were much lower (below 60%) in most instances [43]. McKnight et. al compared the diagnostic code in the physician billing claims for chronic obstructive pulmonary disease (COPD) with the medical chart. They found a sensitivity of 93% to detect COPD and specificity of 94% to 11

25 distinguish COPD from asthma [37]. Beitel et al. compared diagnostic codes for acute pediatric respiratory illnesses with the medical chart. These authors found a specificity of 98%, and a sensitivity of 70% for the correct classification in the claims database [32]. Muhajarine et al. compared the diagnostic code for hypertension to actually measured blood pressure in the medical chart. A kappa of 0.6 was found as the overall level of agreement [39]. Bernatsky et al. demonstrated using diagnostic codes from a physician claims database and hospital admission database to study incidence and prevalence of systemic lupus erythromatosus found that estimates varied greatly depending on the approach to case ascertainment [44]. They attempted to use statistical methods to account for inherent misclassification error. Inputting a diagnostic code is often challenging and subject to interpretation. Multiple diagnoses of varying importance can simultaneously exist in the same patient. Yet, only one diagnostic code per physician claim can be entered. This may partly explain the relatively low levels of agreement illustrated by the sensitivities and specificities in the 5 above studies Studies Using the Procedure Code Physician claims procedure codes represent the exact procedure or act the physician performed for which he/she is to be paid. Therefore, of all the types of physician claims data, these are presumed to have a very high level of agreement when comparing the database to the patient medical chart [38]. Two studies that attempted to validate the procedure codes are as follows: Tamblyn et al. examined injuries in the elderly relating to falls by comparing procedure codes in physician billing claims with patient medical charts. Using the procedure codes alone to assess injuries, the sensitivity of the database for detecting any injury ranged from 27-54% [42]. However, although this study used procedure codes, they were used to infer the diagnosis and not the procedure itself. Therefore, this study was not truly validating the procedure codes themselves. A systematic review of discharge coding from the UK was performed by Campbell et al., who found a median accuracy rate for procedural codes of 98% in Scotland and 70% in England [45]. However, this study did not clarify which proportion of these procedure codes (if any) were 12

26 actually linked to physician claims. Therefore, this study did not in fact validate physician claims database procedure codes. Ladouceur et al. combined diagnostic codes, prescription claims and procedure codes in order to define prevalence estimates for osteoarthritis, and found that despite using varying models and assumptions, prevalence estimates varied widely [46]. They caution against using administrative databases without validation studies Database Research in the Gastroenterology Literature In the gastroenterology literature, a similar spectrum of database studies exist. Hospital administrative databases have been used to examine peptic ulcer disease and gastrointestinal bleeding [47]. Studies have once again used both diagnostic codes and procedure codes in physician claims databases. For example, in British Columbia using their medical services plan database, enteric illness was examined using both diagnostic codes and procedure codes (referred to as fee items). Although the medical chart was not used to validate the data, the authors found that an analysis based on the fee items was more accurate than one based on diagnostic codes [48]. One study in North Carolina validated procedure codes for colonoscopy and flexible sigmoidoscopy as recorded in the physician claims database (Medicare) with patient medical records [49]. The medical records targeted by the authors were the medical records of the primary care physician, not the patient medical records of the physician who submitted the claim. The information abstracted from the medical records were used interchangeably with the physician claims records as the gold standard. Interestingly 32% of all procedures in the medical record had no date associated with them and abstractors used other evidence to determine if colonoscopies had occurred during the study window. Twelve percent of patients enrolled did not have data successfully abstracted and were excluded. With the medical record as the gold standard for colonoscopy, the Medicare claims had a sensitivity of 94%, a specificity of 95%, a PPV of 91%, a NPV of 97%, a concordance of 95%, and a Kappa of

27 In Canada, procedure codes for gastroscopy, colonoscopy, flexible sigmoidoscopy, and polypectomy have been used to assess population procedure rates and resource utilization. The Ontario Health Insurance Plan, the RAMQ, and the Alberta Health Care Insurance Plan have been used numerous times for these purposes [50-54]. However, no Canadian study sought to validate the use of these procedure codes. Diagnostic and procedure codes are also used outside of physician claims by examining discharge summaries from hospital admissions. One Canadian study compared the medical chart to discharge summaries for many different diagnostic and procedure codes [55]. They found significant differences in coding for gastroscopy and colonoscopy between the medical chart and discharge summaries. 14

28 3 OBJECTIVES and HYPOTHESES 3.1 Main Objective The main objective of this thesis is to determine the level of agreement between RAMQ physician claims database procedure code for polypectomy (0749) and polypectomy documentation in the endoscopy report (medical chart) of patients who underwent colonoscopy in Montreal (Figure 2). All endoscopists fill out the endoscopy report immediately after the procedure. The endoscopy report is part of the medical chart. The medical chart has been used numerous times as the gold standard for administrative database research [32, 37, 42, 43, 49]. Predictors: Institution Specialty Indication Physician Figure 4 - Model of Main Objective and Objective #2 3.2 Secondary Objectives There are 3 secondary objectives. Objective #2: To determine predictors of the level of agreement between RAMQ physician claims database procedure code for polypectomy (0749) and polypectomy documentation in the endoscopy report (medical chart) of patients who underwent colonoscopy in Montreal (Figure 2). 15

29 We hypothesize that predictors of the level of agreement between the 2 data sources will reflect a characteristic specific to the endoscopist or environment in which the colonoscopy was performed. It is unlikely that patient characteristics would influence proper recording or coding of a polypectomy with the possible exception of indication for colonoscopy. Specific predictors selected to examine objective #2 (Figure 2) were four variables collected from the endoscopist s questionnaire (Appendix 3): institution where colonoscopy was performed, specialty of the endoscopist (physician), indication for colonoscopy and individual endoscopist. Objective #3: To estimate and compare our study findings with the published literature for: a) percentage of colonoscopies done for screening purposes and b) percentage of screening colonoscopies that result in a polypectomy. Objective #4: To estimate: a) the annual number and cost of screening colonoscopies in Quebec in 2007, b) the annual number and cost of polypectomies resulting from screening colonoscopies in Quebec in

30 4 METHODOLOGY 4.1 STUDY DESIGN Overview Our study is a secondary analysis of a prospective cohort study aimed at developing an algorithm to identify screening colonoscopies in the RAMQ database. All participating patients and endoscopists in our retrospective study were participants in this prior cohort study. Participants were patients who underwent colonoscopy in Montreal from January to March Both patients and endoscopists completed brief questionnaires on the indication for the colonoscopy Study Sites Seven institutions that participated in the original prospective cohort study were included in our retrospective study: Montreal General Hospital, Royal Victoria Hospital, St. Mary s Hospital Centre, Jewish General Hospital, Hôtel Dieu, Hôpital Maisonneuve-Rosemont, and Hôpital Fleury. Four are teaching hospitals affiliated with McGill University, 2 are teaching hospitals affiliated with Université de Montréal and one is a community hospital. 4.2 STUDY POPULATIONS Endoscopists Inclusion criterion for endoscopists in the original cohort study was that endoscopists perform colonoscopy at one of the participating study sites. Our study added the exclusion criterion: endoscopists not eligible to submit physician claims to the RAMQ for remuneration (endoscopists on a different fee schedule or who were salaried) Patients Inclusion criteria for patients in the original cohort study were: men and women aged 50 to 80 who underwent a scheduled colonoscopy in Montreal from January to March Exclusion criteria for patients in the original cohort study were: 1) not eligible for coverage by the provincial health insurance plan in the previous one year and 2) unable to provide informed consent. 4.3 PROCEDURE 17

31 Data from the original cohort study were collected from 4 independent sources: 1) patient questionnaire, 2) endoscopist questionnaire 3) endoscopy report 4) physician claims records (RAMQ database) Endoscopist Questionnaire Endoscopists had been approached prior to study inception and explained the research protocol. Informed consent was obtained (Appendix 2), and the RA attached a brief questionnaire to the outside of a participating patient s medical file that was completed by the endoscopist at the time of index colonoscopy (Appendix 4). The endoscopist questionnaire was used to provide the specialty of the endoscopist, the indication for colonoscopy, and the institution where the procedure was performed Patient Questionnaire Prior to undergoing the index colonoscopy, the research assistant (RA) approached the patient, explained the study, obtained written informed consent (Appendix 3), and interviewed the patient, using a 10-item questionnaire (Appendix 5) on demographic information and gastrointestinal past medical history Endoscopy Report In the present study, patient medical charts that contain the endoscopy report for colonoscopy were reviewed by 2 members (M.S. and J.W.) of the research team who were blinded to RAMQ database polypectomy status. For each subject, information regarding polypectomy performance was abstracted (Appendix 6). We considered polypectomy data in the endoscopy report as the gold standard for polypectomy performance during index colonoscopy Physician Claims Records Physician claims records from the day of index colonoscopy were obtained using patients health care numbers from the RAMQ. Physician claims records were used as a second source of data to determine if participants underwent polypectomies. Physician claims records were also obtained for each patient for 2 months prior and subsequent to the index colonoscopy. 18

32 4.4 VARIABLES Patient Questionnaire Variables collected from the patient self-reported questionnaire included patient age (continuous; 50 to 80 years of age) and sex (binary; male, female) Endoscopist Questionnaire Variables collected from the endoscopist self-reported questionnaire included location of colonoscopy (nominal; each hospital is coded for by a letter: A, B, C, D, E, F, G), specialty (binary; gastroenterologist, non-gastroenterologist), and indication for colonoscopy (binary; screening, non-screening). Screening colonoscopies were defined as either average-risk (no endorsing reason other than age 50 to 80), or having a family history of CRC (1st degree relative with CRC). Non-screening colonoscopies were defined as all other indications Endoscopy Report One variable was collected retrospectively from the medical chart containing the endoscopy report: Polypectomy performed during index colonoscopy (binary; yes, no) was recorded Physician Claims Database Two variables were collected from the RAMQ physician claims database. Presence of the polypectomy procedure code (0749) for index colonoscopy (binary; yes, no) and presence of the colonoscopy procedure code (0697) for index colonoscopy (binary; yes, no) were recorded. 4.5 STATISTICAL ANALYSIS Main Objective: Validity of Polypectomy Procedure Code Presence of a polypectomy performed at index colonoscopy as recorded in the endoscopy report (gold standard) was compared to presence of polypectomy procedure code for index colonoscopy in the RAMQ physician claims database using the following statistical methods: 19

33 Basic descriptive statistics were used to characterize the patient population. Subsequently, specific parameters including sensitivity, specificity, positive predictive value, negative predictive value, as well as the concordance and kappa statistic were calculated as follows (Table 3) [56, 57]: For the purposes of this study: Sensitivity is the proportion of colonoscopies with a polypectomy recorded in the endoscopy report that had a polypectomy procedure code in the RAMQ database (true positives) [a/(a+c)]. Specificity is the proportion of colonoscopies without a polypectomy in the endoscopy report that did not have a polypectomy procedure code in the RAMQ database (true negatives) [d/(b+d)]. Positive predictive value (PPV) is the proportion of colonoscopies with a polypectomy procedure code in the RAMQ database that had a polypectomy recorded in the endoscopy report [a/(a+b)]. Negative predictive value (NPV) is the proportion of colonoscopies without a polypectomy procedure code in the RAMQ database that did not have a polypectomy recorded in the endoscopy report [d/(c+d)]. Concordance is the degree of inter-rater agreement between having a polypectomy recorded in the endoscopy report and the presence of a polypectomy procedure code in the RAMQ database [(a+d)/(a+b+c+d)]. Discordance was defined as the sum of false positives and false negatives divided by the total (b+c/a+b+c+d). Kappa statistic, another measure of inter-rater agreement is calculated as: (observed agreement - chance agreement)/(1 - chance agreement), where observed agreement = concordance, and estimated chance agreement = [(a+b)/(a+b+c+d)(a/(a+b+c+d)] + [(b+d)/ (a+b+c+d)(d/(a+b+c+d)] (Table 3). Precision of our results is presented with 95% confidence intervals in square brackets (or 95% credible intervals when a Bayesian approach is used). 20

34 Table 3: Contingency Table for Level of Agreement Physician Claims Records Polypectomy Endoscopy Report Polypectomy (Gold Standard) YES NO YES a b a+b NO c d c+d a+c b+d a+b+c+d Projected Precision Estimation for a Fixed Sample Size Our study had a fixed sample size (N=689). Precision estimates were calculated assuming a value of 90% for the sensitivity and specificity. A value of 90% was chosen for two reasons. First, the only study to attempt validation of the colonoscopy procedure code [49] found a sensitivity of 94% and a specificity of 95%. We expected the parameters for polypectomy to be similar. Second, endoscopist remuneration is tied directly to correctly submitting procedure codes; we expected that major errors are unlikely to occur. The U.S. Taskforce [58] suggests an adenoma detection rate for screening colonoscopies of at least 25% for men and 15% for women. Assuming an equal number of men and women in our study, the overall adenoma detection rate is approximately 20%. In addition to adenomas, we assume a 5% rate of hyperplastic polyps. Therefore, we chose a 25% overall polyp detection rate (polypectomy rate). With calculations based on a single binomial, Table 4 displays our projected precision estimates for the parameters sensitivity, specificity, PPV, NPV and concordance. 21

35 Table 4 - Precision Estimates for Fixed Sample Size Parameter Relevant Sample Size Estimates Precision * Sensitivity 689 x 25% (polyp prevalence in cells a+c) = 172 +/ or +/- 4.5% Specificity 689 x 75% (no polyp as found in cells b+d) = 517 +/ or +/- 2.6% PPV 689 x 25% (polyp prevalence in cells a+b) = 207 +/ or +/- 5.9% NPV 689 x 75% (no polyp as found in cells c+d) = 482 +/ or +/- 1.7% Concordance 689 (cells a + b + c + d) +/ or +/- 2.2% * 95% CI Objective #2: Predictors of Level of Agreement Possible predictors of the level of agreement (institution, specialty, indication, endoscopist) were examined by creating 2x2 tables and performing univariate analyses to determine which predictors (if any) substantially alter the level of accuracy (more specifically the level of discordance). Subsequently a multivariate logistic regression analysis was performed if strong predictors are found Objective #3: Comparison of our Study with the Literature Based on the literature review (sections and 2.1.6) we estimated the percentage of colonoscopies done for screening purposes and the percentage of screening colonoscopies with polypectomy. We compared these estimates with our study findings Percentage of Colonoscopies Done for Screening Purposes In order to determine the expected proportion of colonoscopies performed for screening to the proportion found in our study, a literature review was performed (section 2.1.5). Of 6 studies [18-23] that met our 4 criteria, 3 had substantial exclusions. Therefore, a sensitivity analysis was done with one analysis utilizing all 6 studies and a second analysis using only data from 3 studies without substantial exclusions [19, 20, 23]. These two analyses, and a third analysis based on our own study data, were performed by fitting a Bayesian hierarchical (random effects) model [59] (Appendix 7). Proportions and credible intervals were calculated for the mean. Similarly, proportions and credible intervals were calculated predicting the next expected study (literature review) or next expected endoscopist (our study). Next expected refers to a 22

36 statistical prediction for an additional random study or endoscopist. Resulting proportions from the two sources of information were then compared Percentage of Screening Colonoscopies with Polypectomy In order to compare the expected proportion of screening colonoscopies with polypectomy to the proportion found in our study, a literature review was performed (section 2.1.6). Four studies met our criteria [28-31]. A Bayesian hierarchical (random effects) model [59] (Appendix 7) was fitted to these studies and our study data. Table 1 was simplified to contain information on polypectomy rates only, in order to be directly compared to our study results which do not have information on histology. Proportions and credible intervals were calculated for the mean. Similarly, proportions and credible intervals were calculated predicting next expected study (literature review) or next expected endoscopist (our study). Resulting proportions from the two sources of information were then compared Objective #4: Annual Procedure Frequency and Costs In order to estimate annual procedure frequency and costs for screening colonoscopies with at least one polypectomy, total number of complete colonoscopies performed during 2007 was obtained from the RAMQ. We also acquired endoscopist fees associated with colonoscopies and polypectomies in From our study findings, we have estimates for the following: 1) proportion of total colonoscopies performed for screening purposes, 2) proportion of complete (successful) colonoscopies performed and 3) proportion of polypectomies performed during screening colonoscopies. Combining the 2 sources of information (RAMQ data and our study data), frequencies and costs of screening colonoscopies and polypectomies in Quebec for 2007 were estimated as follows: Step #1 Total number of colonoscopies in Quebec in 2007 (a). Total number of colonoscopies (procedure code 0697) was obtained directly from the RAMQ. Step #2 Total number of attempted colonoscopies in Quebec in 2007 (b): Total number of attempted colonoscopies was estimated by dividing total number of colonoscopies (a) by 23

37 proportion of complete colonoscopies in our study. Where [(b) = (a) (total complete colonoscopies in our study total attempted colonoscopies)]. Step #3 Total number of attempted colonoscopies with screening as the indication in Quebec in 2007 (c): Total number of screening colonoscopies was estimated by multiplying total number of attempted colonoscopies (b) by proportion of screening colonoscopies in our study. Where [(c) = (b) x (total screening colonoscopies in our study total attempted colonoscopies)]. Step #4 Total number of completed screening colonoscopies in Quebec in 2007 (d): Total number of completed screening colonoscopies was estimated by multiplying total number of attempted colonoscopies with screening as the indication (c) by proportion of successfully completed screening colonoscopies in our study. Where [d = c x (total complete screening colonoscopies total attempted screening colonoscopies)]. Step #4a Actual cost of completed screening colonoscopies WITHOUT polypectomy (e): Endoscopist fee per screening colonoscopy in 2007 was approximately $100. Total cost of completed screening colonoscopies (e) is [(d) x 100]. Step #5 Total number of screening colonoscopies with at least one polypectomy performed in Quebec in 2007 (f). Total number of screening colonoscopies with at least one polypectomy (f) was estimated by multiplying total number of completed screening colonoscopies (d) by proportion of screening colonoscopies that had a polypectomy in our study. Where [(f) = (d) x (total screening colonoscopies with polypectomy total attempted screening colonoscopies)]. Step #5a Actual cost of polypectomies during screening colonoscopy (g): Endoscopist fee per polypectomy in 2007 was approximately $15. Total cost of at least one polypectomy during completed screening colonoscopies (g) is [(f) x 15]. Total cost does not include additional endoscopist fees if >2 polyps are removed. Step #6 Total cost of screening colonoscopies and polypectomies (h). Total cost is the sum of [(e) + (g)]. 24