Approximately 20,000 influenza-associated deaths occurred

Self-Reported and Claims-Based Influenza Immunization Rates for Medicare Beneficiaries in Montana and Wyoming, Michael J. McInerney, PhD, M liss A. Markham, MS, Martin J. Kileen, MD, and Steven D. Helgerson, MD Objective: To determine the self-reported influenza vaccination rate for Medicare beneficiaries in Montana and Wyoming and to compare it with the rate based on Medicare billing data. Design: Telephone survey. Setting and participants: A random sample of Medicare beneficiaries aged 65 years and older living in Montana and Wyoming (n = 1590). Measures: Self-reported and claims-identified influenza immunization rates. Results: Interviews were completed with 841 beneficiaries from the sample. Of these, 71% reported that they had received an influenza vaccination; only 73% of those who reported being vaccinated had a corresponding Medicare billing claim. Compared with selfreported influenza immunization data, billing claims had a sensitivity of 76%, specificity of 97%, positive predictive value of 98%, and a negative predictive value of 38%. Conclusion: Medicare billing claims data substantially underestimate the actual influenza immunization rate; more accurate self-reported rates can be generated in a more timely manner. In the future, surveillance of influenza immunization rates in Medicare beneficiaries should be based on timely, area-wide self-reported data. Approximately 20,000 influenza-associated deaths occurred during each of the 9 influenza epidemics in the United States between 1972 73 and 1991 92 [1]. More than 90% of the deaths attributed to pneumonia and influenza occurred among persons aged 65 years or older [1]. Annual influenza vaccination for all persons aged 65 years and older has been recommended to reduce the mortality and morbidity associated with influenza in this population [1 3]. A national health objective for the year 2010 is to achieve an influenza immunization level of at least 90% in this group [3]. Letters mailed to Medicare beneficiaries reminding them to get vaccinated have been shown to increase the influenza immunization rate [4,5]. In order to determine where to apply this strategy and commit intervention resources, it is essential to have accurate surveillance. Traditionally, immunization rate estimates are calculated from Medicare billing claims. However, several studies have shown that selfreported influenza immunization rates for persons aged 65 years and older are higher than the claims-identified rates. Self-requested rates for persons in the 1993 94/ 1994 95 National Health Interview Survey were 52% and 55%, compared with rates of 35% and 38% derived from Medicare billing claims [6]. Similarly, in the Medicare Current Beneficiary Survey for 1993 and 1994, the selfreported vaccination rates were 51.5% for white beneficiaries and 31.3% for black beneficiaries, while the rates for whites and blacks from Medicare billing claims were 37% and 17% [7]. In Montana, the self-reported influenza immunization rates for persons aged 65 years or older for 1993 and 1994 were 62% and 65% compared with rates of 48% and 49% derived from Medicare billing claims [8]. In order to better understand the disparity between selfreported and claims-calculated influenza immunization rates for Medicare beneficiaries, the Mountain-Pacific Quality Health Foundation (MPQHF) conducted telephone interviews with a randomly selected group of beneficiaries in Montana and Wyoming. The influenza vaccination rate self-reported by these persons was compared with the rate reflected in Medicare billing claims. We report the results of our assessment in this article. Methods During, the number of Medicare beneficiaries in Montana and Wyoming was 132,814 and 64,993, respectively. Based on the successful mailed reminder project conducted in 1994, the MPQHF sent a letter encouraging influenza From the Mountain-Pacific Quality Health Foundation, Helena, MT (Drs. McInerney and Helgerson and Ms. Markham), and the Health Care Financing Administration, Seattle, WA (Dr. Kileen). www.turner-white.com Vol. 8, No. 5 May 2001 JCOM 31

INFLUENZA VACCINATION Table 1. Characteristics of Respondents and Nonrespondents Beneficiaries, n (%) Both States Montana Wyoming Total N 1590 ( 750 ( 840 ( Respondents* 841 (53) 412 (55) 429 (51) Female 492 (59) 243 (59) 249 (58) Age 65 74 yr 439 (52) 195 (47) 244 (57) Influenza vaccination 446 (53) 250 (61) 197 (46) Nonrespondents 348 ( 169 ( 179 ( (number found but no interview) Female 185 (53) 88 (52) 97 (54) Age 65 74 yr 205 (59) 93 (55) 112 (63) Influenza vaccination 170 (49) 87 (51) 83 (46) Nonrespondents 401 ( 169 ( 232 ( (no telephone number found) Female 246 (61) 95 (56) 151 (65) Age 65 74 yr 202 (50) 75 (44) 127 (55) Influenza vaccination 133 (33) 76 (45) 57 (25) *Percentages in this row are of total sample. immunization in September to all living beneficiaries in both states who were enrolled in the Part B Medicare program, not known to be in institutions such as nursing homes, and not enrolled in a managed care plan. (Medicare reimburses for influenza immunization for beneficiaries enrolled in the Part B program but does not receive billing claims for beneficiaries enrolled in managed care plans.) The number of letters sent was 121,988 in Montana and 44,850 in Wyoming. We identified a random sample of persons who were sent a letter; the sample included 750 persons in Montana and 840 persons in Wyoming. Because current telephone numbers are not included in the Medicare beneficiary files maintained by Health Care Financing Administration (HCFA), we searched residential telephone directories for phone numbers. Telephone interviews were conducted by specially trained MPQHF staff between 20 February and 1 March 1996. The calls were conducted soon after the influenza season to minimize recall bias. Four unanswered calls to a person on different days and times was considered a nonresponse. Persons interviewed were asked if they had received influenza vaccine in autumn. Those who answered yes were asked when (before or after Thanksgiving) and where (private physician s office, health department clinic, a senior center, store, or other site) they received the vaccine. Those who said they were not vaccinated were asked the reason for not being vaccinated; specific responses were categorized into 1 of 15 response category options by analysis. All persons interviewed were also asked if they had received influenza vaccine the preceding year, if they planned to be vaccinated the following year, if they remembered receiving a letter from MPQHF, and if the letter had influenced their decision about whether to get an influenza vaccination. Medicare Part B claims records for 1 September through 31 December were used to identify beneficiaries for whom claims were submitted for influenza vaccination (HCPCS:90724) or influenza vaccine administration (HCPC:G0008, ICD9:VP04.8). Self-reported and claimsidentified influenza immunization histories were then compared for respondents to the telephone survey. Authorization to conduct the telephone survey of Medicare beneficiaries was secured from HCFA. Results A total of 841 beneficiaries in the sample (53%) were interviewed (Table 1). Medicare billing claims for influenza vaccination were submitted for 446 (53%) of these beneficiaries. Compared with the beneficiaries for whom phone numbers were found, beneficiaries for whom no number was found were somewhat less likely to be men (39% versus 43%), to be under 75 years of age (50% versus 54%), and to have had an influenza vaccine claim submitted (33% versus 47%). Seventy-one percent of the beneficiaries for whom phone numbers were found were interviewed. Six hundred of the 841 respondents (71%; 95% confidence interval [CI], 68% to 74%) reported receiving influenza vaccination in autumn (Table 2). Influenza vaccination claims from autumn were submitted for 439 (73%) of the respondents who reported receiving the vaccination, and for 7 (3%) of the respondents who reported not receiving it. Using the self-reported influenza immunization experience in autumn to assess the completeness and accuracy of billing claims, the sensitivity, specificity and positive predictive value of the claims were 76%, 97%, and 98%, respectively. The negative predictive value of the billing claims was 38%. Of the respondents who said they were vaccinated, 82% reported receiving the immunization prior to Thanksgiving, 94% said they had received influenza immunization the previous year, and 95% said they planned to be immunized the following year. Of respondents who reported that they were not vaccinated, 44% said they had been vaccinated the preceding year, but only 13% said they planned to be vaccinated the following season. These respondents were more likely to be women but were similar in age to respondents 32 JCOM May 2001 Vol. 8, No. 5 www.turner-white.com

Table 2. Self-Reported Immunization Status for Autumn Respondents, n (%)* Reported Receiving Reported Not Receiving Influenza Vaccine Influenza Vaccine Total N 600 (71) 240 (29) Female 334 (56) 158 (66) Age 65 74 yr 312 (52) 130 (54) Influenza vaccination 439 (73) 7 (3) *Data for 1 beneficiary who did not remember whether he received the vaccine are not included in this table. Percentages in this row are of total respondents (n = 871). who reported being vaccinated. The main reasons for not receiving influenza vaccination were I never get the flu anyway (24%) and last time I got sick from the shot (19%). The proportion of respondents who remembered receiving a letter from MPQHF (30%) did not differ substantially by state or by self-reported immunization status. Less than 13% of those who did remember receiving the letter reported that the letter helped them decide to get vaccinated. The proportion of influenza vaccination claims submitted in autumn 1994 for the study respondents was similar to the proportion submitted in autumn (Table 3). Respondents who had concordant claim and self-reported histories are compared with respondents with nonconcordant histories in Table 4. The nonconcordant respondents were less likely to report receiving the vaccine from a private physician s office, less likely to have had a Medicare claim submitted for influenza immunization in autumn 1994, and less likely to be female. They were more likely to report receiving the influenza vaccine at a senior center and more likely to be in the 65- to-74 years age category rather than the 75+ years category. Discussion The results of this study are consistent with several previous reports that Medicare billing claims underestimate the proportion of Medicare beneficiaries receiving influenza immunizations [5 7]. In light of evidence that mailed reminders are associated with increased influenza immunization rates in Medicare beneficiaries [4,5], it is important to have accurate and timely immunization surveillance data to guide intervention planning. Such planning could intensify intervention efforts in areas with low rates and redirect intervention resources in areas with high rates. The investigation reported here produced state-specific estimates that were Table 3. Percentage of Respondents for Whom Influenza Vaccination Claims Were Submitted in 1994 and Respondents Who Reported Receiving Influenza Vaccine Respondents Who Reported Not Receiving Influenza Vaccine Influenza vaccination 62% (310/502) 7% (12/178) 1994* Influenza vaccination 76% (439/576) 3% (7/236) *Only beneficiaries who were enrolled in the Medicare Part B Program in the last quarter of 1993 and 1994 were included. Twenty-eight beneficiaries (24 yes respondents, 4 no respondents) were excluded because their health insurance claim identities were in identity sequences that we were unable to equate to the billing health insurance claim numbers. timely enough to use for planning intervention activities in the following influenza season. A major implication of our results concerns the way resources are used to achieve health status goals. At the time of these intervention activities, HCFA and other agencies and groups had adopted the Healthy People 2000 national health objective to increase the influenza immunization rate for persons aged 65 years or older to 60% [2]. Based on selfreported immunization rates, the year 2000 objective was already achieved statewide in Montana and Wyoming and had likely been achieved in other states as well. HCFA and peer review organizations (such as MPQHF) may not need to use resources to maintain an already-achieved goal in some population segments, while attention could be directed to achieving the goal for still unreached subsets of Medicare beneficiaries. The Healthy People 2010 objective to increase immunization rates to 90% in this population underscores the need for timely estimates for defined population subsets [3]. Arelated implication concerns timely program evaluation. The data collection method we used yielded self-reported information about the influenza immunization rate in autumn by March 1996. This information was then available for assessing strategies to increase the immunization rate in the 1996 97 influenza season. Provisional and final data from Medicare billing claims for autumn were not available for use until May and August 1996, respectively, and as described here, systematically underestimated the immunization rate. We feel that the method used in our work could be adapted to provide timely national and regional as well as state-specific influenza immunization estimates for key race and age groups. www.turner-white.com Vol. 8, No. 5 May 2001 JCOM 33

INFLUENZA VACCINATION Table 4. Characteristics of Respondents with Concordant and Nonconcordant Self-Reported and Claim-Based Influenza Immunization Status Respondents, n (%) Characteristics Nonconcordant Concordant Odds Ratio (95% CI) Total N 137 ( 439 ( Reported receiving vaccine before Thanksgiving 107 (78) 368 (84) 1.45 (0.88 2.40) Reported receiving vaccine at: Private physician s office 35 (26) 222 (51) 2.98 (1.91 4.68) Health department 7 (5) 37 (8) 1.71 (0.73 4.65) Senior center 30 (22) 68 (15) 0.65 (0.39 1.08) Store 5 (4) 17 (4) 1.10 (0.37 3.46) Influenza vaccination claim submitted in autumn 1994 32 (23) 267 (61) 5.09 (3.21 8.11) Received letter from MPQHF 41 (30) 142 (32) 1.12 (0.72 1.74) Received influenza immunization before 132 (96) 412 (94) 0.92 (0.29 2.71) Intend to receive influenza immunization next year 132 (96) 415 (95) 1.31 (0.39 4.10) Female 70 (51) 253 (58) 1.30 (0.87 1.95) Age 65 74 yr 77 (56) 225 (51) 0.82 (0.55 1.23) CI = confidence interval; MPQHF = Mountain-Pacific Quality Health Foundation. A third implication concerns the use of Medicare billing claims data for program development and evaluation. Wherever an agency or group chooses to rely on influenza immunization rates derived from claims data as a basis for developing or evaluating an influenza immunization program, efforts to improve the completeness of the billing claims will be needed. In Montana and Wyoming, we found that persons for whom self-report and billing claims data were not concordant for were less likely to have had a billing claim submitted in 1994 and somewhat more likely (not quite reaching the traditional level of statistical significance) to report receiving the immunization at a senior center (Table 4). In these states, efforts to improve completeness of influenza immunization billing claims could begin at senior centers or with beneficiaries for whom no claim was submitted in a previous year. There are several reasons claims are not submitted to Medicare. Beneficiaries may be unaware of Medicare coverage for influenza vaccine. Also, less traditional immunization providers may be less likely to accept Medicare assignment or may offer immunizations for free or at much reduced rates, without claims processing. Finally, there is an opportunity for local agencies or groups to try to achieve health objectives that meet or exceed those set for the nation. In order to achieve these new levels, it would be necessary to immunize at least some of the persons who are now persistent nonvaccinees. More than half of those who did not get influenza vaccination reported, I never get the flu anyway, last time I got sick from the shot, or don t believe in it. Less than 5% reported they forgot to or meant to get one. It is likely that innovative strategies will need to be developed if a substantial proportion of current nonvaccinees is to be reached. These results should be interpreted with caution. Surveys conducted several months after the health care encounter are subject to recall bias [9]; however, the magnitude of this bias may be inconsequential when considering the increased timeliness of the data. Also, we were not able to find phone numbers for 25% of the Medicare beneficiaries sampled, nor were we able to complete interviews with 29% of those for whom phone numbers were identified. The beneficiaries without an identifiable phone number were less likely to have had a billed vaccine claim in 1994 than were beneficiaries with identifiable phone numbers. An alternate method, perhaps the use of mailed questionnaires [5], will be needed to monitor the immunization rate for the former group of beneficiaries. Thus, the interview results reported here come from 53% of the original sample and may not be entirely representative of Medicare beneficiaries in the 2 states. On the other hand, the 71% influenza immunization rate reported by these telephone respondents is consistent with the rates (64% and 71% in Montana; 64% and 67% in Wyoming) reported by respondents identified as age 65 or older through the telephone survey in Montana and Wyoming in and 1996 [8]. Thus, the results reported here probably reflect information pertinent at least to older persons with telephones. The results of this assessment suggest that the effect of contacting individual beneficiaries by mail as done in Montana and Wyoming is likely to be limited by the proportion of beneficiaries who are resolute about not receiving influenza vaccine. However, while there may be a ceiling 34 JCOM May 2001 Vol. 8, No. 5 www.turner-white.com

to the attainable immunization rate [10], it is likely that the current rate can be increased further. An individual immunization history is the greatest predictor of future immunization, and once activated, may not require yearly mail contact. It is also apparent that use of limited-scope beneficiary surveys are beneficial in assessment of interventions both in more timely assessment than assembling billing data, and in collection of independent factors associated with the measured response. The authors thank Peg Donahue, Christy Fuller, Roni Jean, Sara Medley, Donna Small, Fran VanNatta, Belva Verzatt, and Christi Williams who conducted phone interviews; Ernest Kimball, MPH, for his assistance in training the interview team; and the Medicare beneficiaries in Montana and Wyoming who responded to our questions. Corresponding author: Michael J. McInerney, PhD, Mountain-Pacific Quality Health Foundation, 3404 Cooney Dr., Helena, MT 59602, MTPRO.mmcinerney@sdps.org. The analyses upon which this publication is based were performed under contract numbers 500-96-P507 (Montana) and 500-96-P508 (Wyoming) of the Health Care Quality Improvement Program initiated by the Health Care Financing Administration and required no special funding on the part of the contractor. Author contributions: conception and design, MJM, MAM, MJK, SDH; analysis and interpretation of data, MJM, MAM, MJK, SDH; drafting of the article, MJM, MAM, MJK, SDH; critical revision of the article, MJM, MJK, SDH; final approval of the article, MJM, MAM, SDH; provision of study materials or patients, MJM, SDH; statistical experience, MJM, MAM, SDH; administrative, technical, or logistic support, MJM, MJK; collection and assembly of data, MJM, MAM, MJK, SDH. References 1. Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morb Mortal Wkly Rep 1997;46:1 25. 2. Healthy people 2000: national health promotion and disease prevention objectives. Washington (DC): U.S. Dept. of Health and Human Services, Public Health Service; 1991. 3. Healthy people 2010. Washington (DC): U.S. Dept. of Health and Human Services, Public Health Service; 2000. Available at http://www.health.gov/healthypeople. 4. Increasing influenza vaccination rates for Medicare beneficiaries Montana and Wyoming, 1994. MMWR Mortal Morb Weekly Rep ;44:744 6. 5. Smith DM, Zhou XH, Weinberger M, et al. Mailed reminders for area-wide influenza immunization: a randomized controlled trial. J Am Geriatr Soc 1999;47:1 5. 6. Influenza and pneumococcal vaccination coverage levels among persons aged 65 years United States, 1973 1993. MMWR Mortal Morb Weekly Rep ;44:506 7, 513 5. 7. Gornick ME, Eggers PW, Reilly TW, et al. Effects of race and income on mortality and use of services among Medicare beneficiaries. N Eng J Med 1996;335:791 9. 8. Assessing health risks in Montana, 1994 report. Helena (MT): Montana Dept. of Public Health and Human Services; 1997. 9. Toy GM, Belli RF, Lepkowski JM. Quality of last doctor visit reports: a comparison of medical record and survey data. In: American Statistical Association 1994 Proceedings of the Section on Survey Research Method. Washington (DC): 1994; 362 72. 10. Buchner DM, Larson EB, White RF. Influenza vaccination in community elderly. A controlled trial of postcard reminders. J Am Geriatr Soc 1987;35:755 60. Copyright 2001 by Turner White Communications Inc., Wayne, PA. All rights reserved. www.turner-white.com Vol. 8, No. 5 May 2001 JCOM 35