DataWatch. Using Survey Data To Estimate Prescription Drug Costs by Marc L. Berk, Claudia L. Schur, and Penny Mohr

Transcription

1 DataWatch Using Survey Data To Estimate Prescription Drug Costs by Marc L. Berk, Claudia L. Schur, and Penny Mohr After Congress passed the Medicare Catastrophic Coverage Act of 1988 (PL ), interest in the accuracy of estimates of prescription drug expenditures increased dramatically. While any attempt to model the cost of a drug benefit depends on establishing an accurate baseline for national prescription drug expenditures, estimating such a baseline has resulted in considerable controversy. Early estimates of the cost of the proposed drug benefit ranged from a low of $2.0 billion by the Congressional Budget Office (CBO) to a high of $6.8 billion by the Health Care Financing Administration (HCFA). 1 These estimates relied on databases developed from surveys administered between 1967 and With the completion of the 1987 National Medical Expenditure Survey (NMES), a timely and nationally representative database became available. 2 However, there remained disagreement among researchers within the Department of Health and Human Services (HHS) about the appropriateness of using data based on household reports to make national expenditure estimates. A critical aspect of the disagreement focused on the extent of misreporting of expenditures by household respondents. There was little empirical evidence on the accuracy of household reporting of prescribed medicines in NMES, although use of services other than prescription drugs was verified by provider reports. Thus, concerns about the reliability of household reporting made projections of the cost of the drug benefit in the Medicare catastrophic legislation difficult. For this reason, the Prescription Drug Expenditure Verification Survey (PDEVS) was designed by Project HOPE to assess the accuracy of reports by the elderly on their use of and expenditures for prescription drugs. The collection of expenditure information from both Marc Berk, a sociologist, is deputy director of the Project HOPE Center for Health Affairs (CHA) in Chevy Chase, Maryland. Claudia Schur, an economist, is a senior policy analyst at CHA; Penny Mohr, also an economist, is a policy analyst While at the National Center for Health Services Research, Berk and Schur participated in the design and implementation of the 1987 National Medical Expenditure Survey (NMES).

2 DATAWATCH 147 households and their pharmacies allowed verification of household reports and inferences about the reliability of data collected in a household survey for making national estimates. Prior Evidence On The Accuracy Of Household Reports There are several reasons why respondents may not be able to report expenditures for prescription drugs. John Moeller and Nancy Mathiowetz report that, in NMES, "{i}tem nonresponse resulted from two types of respondents: those who knew the charges at some point but were unable to recall them at the time of interview and those who were never aware of the charges." 3 In addition, they report that some respondents "with private insurance reported the copayment amount as the total charge." In NMES, 41 percent of the expenditures for elderly persons had to be imputed. In May 1989, HHS submitted a report to Congress entitled Expenses Incurred by Medicare Beneficiaries for Prescription Drugs, the main body of which was written by researchers in the HCFA Office of the Actuary. Although one of the purposes of the 1987 NMES was to serve as a tool in evaluating new policy measures, HCFA researchers concluded that it was inappropriate to use NMES data to estimate total prescription drug spending by Medicare enrollees: Like other household data, NMES is subject to response error and bias that make it impossible to derive aggregate figures directly The strength of a population survey lies in its ability to relate spending with sociodemographic and economic factors, and it is in these areas that the survey results were used. However, population surveys also reflect errors of recall and omission on the part of respondents, a fact confirmed by surveys previous to NMES and explained in greater detail in... the technical appendices to... this report. Consequently, reliance upon NMES alone to estimate the level of expenditure for prescription drugs will lead to an underestimate of actual experience. 4 While HCFA's report cast doubt on NMES's usefulness in making national estimates, the technical appendices came to a different conclusion. In those appendices, staff from the National Center for Health Services Research and Health Care Technology Assessment (NCHSR) compared NMES estimates with those obtained from a proprietary database and concluded that there "appears to be no consistent underreporting of prescriptions for the Medicare population" and that "there is no evidence of charges in NMES being consistently lower or higher than those reported by [pharmacies]." 5 In October 1989, CBO released its report, Updated Estimates of Medicare's Catastrophic Drug Insurance Program. As did HCFA, CBO concluded that underreporting on NMES was a potentially significant prob-

3 148 HEALTH AFFAIRS Fall 1990 lem. In addition, CBO concluded that "the data necessary for convincing validity checks do not currently exist." 6 In the absence of any recent empirical data quantifying the magnitude of the underestimate, CBO "adjusted upward by 10 percent to compensate for assumed underreporting of drug expenses in this survey." Further CBO analysis estimated that an assumption of 20 percent underreporting, instead of the 10 percent used, would increase fiscal years outlays by $1.8 billion. We are aware of only two efforts to examine the relationship of household-reported data on use and expenditures for drugs with data contained in pharmacy records. Results from the Medical Economics Survey (MES) indicate that household-reported data concerning drug expenditures is subject to significant underreporting. 7 The MES was conducted in two Maryland cities; in Baltimore, household-reported expenditure estimates were approximately 35 percent lower than those obtained from the pharmacies, while in the Hagerstown area, expenditures from the household survey were underreported by about 22 percent. The underestimate for prescribed drugs appears to be considerably greater than for physician outpatient visits, dental care, or hospital visits. 8 NCHSR researchers used two approaches to explore the accuracy of household reporting on prescription drugs. The first was to compare NMES estimates with those obtained from a proprietary database maintained by Pharmaceutical Data Services (PDS). Comparisons between PDS and NMES suggested "an overall underreporting rate of approximately 20 percent." 9 Nonetheless, the analysts suggested that there is no consistent underreporting by Medicare beneficiaries. The NCHSR analysis compared the use of specific drugs most likely to be used by the elderly, rather than charges for all drugs purchased by the elderly. Furthermore, all of the drugs mentioned are used substantially by the nonelderly, and, indeed, nine of the twenty drugs examined by NCHSR are primarily used by nonelderly. While such an analysis is useful in determining the relative accuracy of reporting for certain drugs, it cannot be used to assess the accuracy of drug expenditure reporting by the elderly. NCHSR's Moeller and Mathiowetz conducted a second study on validation of prescription drug estimates. 10 This study examined a subset of NMES cases that could be matched to data from a proprietary claims service. The authors caution that this matched sample "in no way reflects the coverage or representativeness of the full NMES sample." Comparisons indicated that approximately 36 percent of the NMES respondents underreported expenditures for prescription drugs, a little more than twice the percentage who overreported their use of medicine. The study was not limited to the elderly and included only persons with private

4 DATAWATCH 149 insurance coverage for prescription drugs; this population may tend to be more educated, have higher incomes, and also be more likely to maintain accurate records of drug expenditures than the overall population. Policy Implications Of Underreporting By Households Although the Medicare Catastrophic Coverage Act has been repealed, discussion of a prescription drug benefit remains of policy interest. As health care costs rise and the elderly population grows, the benefits provided by the Medicare program as well as services not covered by Medicare are subject to continued scrutiny. Since prescription drugs constitute a significant expenditure by the elderly (a mean expenditure of $253 per person as reported in the 1987 NMES) and are not currently covered on an outpatient basis, their inclusion in the list of benefits will remain a topic of discussion. In fact, several key members of Congress plan to reintroduce related legislation in the next session. To fully understand the cost implications of such legislation, policy analysts must have accurate and reliable data on prescription drug expenditures by the elderly. Starting with an estimate of expenditures in the year preceding the policy change, future drug expenditures could be projected, taking into account expected price inflation and trends in the per capita use of prescribed medicines. A confounding factor in this estimating process is the size of the insurance effect, or induced demand. With respect to measuring the impact of the catastrophic legislation, policy analysts are concerned with a potential increase in prescription drug spending attributable to the change in insurance coverage. The presence of induced demand makes accurate knowledge of expenditures in the years preceding such a major policy change even more critical. Without such information, any predictions about future expenditures would be flawed. From a policy perspective, higher-than-expected increases in program costs would be attributed to the effect of increased insurance coverage, when, in fact, the artificially low baseline estimate was the true culprit. Furthermore, from a budgetary perspective, the lack of a reliable baseline estimate of expenditures means that the reported effect on program costs would be biased even after the changes are in place. Study Methods Design of the Prescription Drug Expenditure Verification Survey (PDEVS). Estimates from the 1987 NMES indicate that Medicare beneficiaries age sixty-five or older spent a total of $6.9 billion on prescription

5 150 HEALTH AFFAIRS Fall 1990 drugs. The PDEVS was designed to determine the accuracy of that estimate and of other estimates that may be produced from future surveys that rely on the ability of elderly respondents to accurately recall prescription drug expenditures. The PDEVS, conducted between April and December of 1989, involved interviews with elderly persons in five locations using questionnaires and survey procedures as similar as possible to those used in NMES. The PDEVS included all of the verbatim questions used in NMES concerning use of and expenditures for prescription drugs. Also included were verbatim questions from NMES concerning other types of medical care that might be important in helping respondents to recall their use of prescription drugs, as well as questions concerning disability days and health insurance. 11 Unlike NMES, the PDEVS included a verification component for prescription drug expenditures. After the second phase of interviewing was complete, surveyors contacted all pharmacies to obtain information on the prescription drugs purchased by each respondent during the study reference period. Cumulative response rates on the PDEVS that is, the proportion of people completing the household survey and the permission form and for whom verification data were obtained for all providers and successfully matched to household reports are believed to be generally comparable to those of NMES for the expenditure components that were verified. 12 Development of a correction algorithm. One of the purposes of the PDEVS was to provide a statistical basis for "correcting" householdreported survey data (that is, reducing the error from inaccurate reporting). By developing a multivariate model based on a relationship between pharmacy-reported charges and household-reported expenditures and a variety of personal characteristics, we hoped to explain a significant portion of the bias in reporting. Findings Of The Verification Survey Accuracy of household reporting. Findings from the PDEVS indicate that over 24 percent of respondents who failed to report any prescription drug expenditures had, indeed, purchased drugs as indicated by their pharmacy record. Also, in 4 percent of cases, a household reported drug expenditures, but the pharmacy did not confirm that any prescriptions had been filled. To provide an overview of the accuracy of household reporting in the PDEVS, we calculated the ratio of semiannual household-reported expenditures to pharmacy-reported charges. Of the 385 persons who used

6 DATAWATCH 151 prescribed medicines during the reference period, fewer than 14 percent of respondents were able to report their expenditures within 10 percent of the actual charges. Approximately 56 percent of respondents underreported the amount spent on drugs by more than 10 percent, while about 29 percent of respondents overreported by more than 10 percent. Whites, persons with private insurance, and persons with twelve or more years of education were more likely than other persons to report accurately (within 10 percent). Exhibit 1 shows a comparison of the mean semiannual expenditures reported by households alongside the pharmacy-reported charges. Overall, there was a 23 percent level of underreporting of expenditures on prescription drugs in the survey population. In dollar terms, the mean household-reported expenditure was $140 compared to $181 for the pharmacy. Exhibit 1 Comparison Of Household And Pharmacy Reports, Mean Dollar Amounts And Percent Underreported, 1989 Characteristics Total Age years years 80 years and over Race White Nonwhite Sex Male Female Health insurance Private No private Health status Excellent/good Fair/poor Education Fewer than 12 years 12 years 13 years or more Number of persons Mean household report $ Mean pharmacy report $ Source: Project HOPE Prescription Drug Expenditure Verification Survey, a Significant at.01. b Significant at.10. c Significant at.05. Absolute difference $ Percent underreported 22.9% a a 22.4 b 21.2 a 31.2 e 28.0 a 19.8 e 21.0 a 29.5 e 20.3 e 25.3 a 26.7 a 18.6 e 21.6 b

7 152 HEALTH AFFAIRS Fall 1990 The level of underreporting varied systematically with a number of demographic characteristics. As one might predict, the younger elderly (under age seventy) reported most accurately. Although the relationship with age appears to reverse direction, this may be due to proxy reporting for the oldest group. Both racial and gender differences can be seen, with nonwhites and males underreporting more than whites and females, respectively. Persons with private insurance (either alone or in addition to public coverage) tended to report more accurately; this is likely attributable to the act of filing claims to receive reimbursement for expenses. Increased years of education also seem to be related to more reliable reporting. Furthermore, those in fair or poor health underreported somewhat more than those in better health, although this may be linked to age. Results of the correction algorithm. The dependent variable in the final correction model is annual per person charges for prescribed medicines as reported in the verification survey of pharmacies; the model includes the following independent variables: household-reported prescription drug expenditures; square of household-reported prescription drug expenditures; the number of prescriptions obtained as reported by the household; perceived health status; age; race; and years of education. 13 This model accounts for 58 percent of the variation in pharmacyreported charges. Using the estimated coefficients obtained in the model described above, data from a household survey can be reestimated to get a more accurate picture of expenditures. Although data from a study in twenty neighborhoods in five cities, such as the PDEVS, would not normally be applied to a nationally representative database such as NMES, it is clear that the options for estimating aggregate expenditures on prescription drugs are severely constrained. Both HCFA and CBO have unequivocally stated that it is not appropriate to use NMES in its current form to make national cost projections; on the other hand, another data source with verified prescription drug expenditures will not be available for at least several years. Thus, we have used the correction algorithm developed with the PDEVS data to adjust the household-reported prescription drug expenditures of elderly Medicare beneficiaries as collected in NMES. Policy Implications Of The Corrected Estimates Exhibit 2 provides estimates of expenditures before and after application of the PDEVS correction model. In the aggregate, expenditures on prescribed medicines by elderly Medicare beneficiaries as reported in

8 DATAWATCH 153 Exhibit 2 Annual Expenditures For Prescribed Medicines Obtained By Noninstitutionalized Elderly Medicare Beneficiaries, 1987 Total Age and over Sex Male Female Race/ethnicity White Black Hispanic Number of prescriptions Oto 5 6 to or more Poverty status Poor Near poor Low income Middle income High income Insurance coverage Any private Public only Number of persons (thousands) 27,421 16,804 8,436 2,181 11,282 16,139 23,829 2, ,578 9,473 7,370 Total expenditures (millions of dollars) NMES Corrected $6,930 $10,571 4,108 2, ,616 4,313 6, ,930 4,728 6,616 3, ,034 6,537 9, ,095 3,190 6,287 Absolute difference $3,641 2, ,418 2,224 3, ,260 1,559 Poor Near poor Low income Middle income High income 3,131 1,876 4,985 8,491 7, ,353 2,113 1,883 1, ,034 3,210 2, ,097 1, Any private 20,585 5,325 8,088 2, Public only 6, ,603 2, Percent underreported 34.4% Source: Project HOPE Prescription Drug Expenditure Verification Survey (PDEVS), 1990; and 1987 National Medical Expenditure Survey (NMES). Note: Expenditures as reported in 1987 NMES and after application of the PDEVS correction algorithm. NMES were $6.9 billion. After applying the algorithm described above, these expenditures increased to $10.6 billion, implying a potential underestimate of 34 percent. The mean expenditure per elderly Medicare beneficiary in 1987 was $253 as calculated from the NMES database and $385 after correction. While expenditures on prescribed medicines varied by individual characteristics, the extent of the underestimate was fairly consistent across demographic groups, with two notable exceptions. The accuracy of reporting appears to increase with age but is likely due to increased use of proxy reporting for older persons. The rate of underreporting according to prescription use is also somewhat deceptive. While the rate of underreporting is very high for persons with fewer prescriptions, this is

9 154 HEALTH AFFAIRS Fall 1990 because the absolute dollar level is low. For persons with over twenty prescriptions, on the other hand, the actual dollar difference between the NMES and corrected estimates is significantly higher than in any other group shown. Although the overall rate of underreporting was 23 percent within the PDEVS, the aggregate and mean expenditure estimates calculated from NMES are 34 percent below the corrected estimates. This is due to the differences in population characteristics between NMES and the PDEVS. Respondents to the PDEVS tended to have higher levels of education and better health status, characteristics associated with more accurate reporting. The multivariate model that we developed takes into account the differences between the PDEVS sample and those obtained from a national sampling frame. We assume that reporting differences are a function of variables such as age, race, sex, education, and health status. Our methodology assumes that living in Philadelphia, Memphis, Council Bluffs, Detroit, or Phoenix per se neither increases nor decreases the ability to report expenditures. Implications for future surveys. Results from the PDEVS clearly indicate that elderly respondents are not able to accurately recall their expenditures for prescription drugs. Underreporting is about twice as prevalent as overreporting. This in turn can lead to significant underestimates in aggregate expenditures. This underestimate appears in varying degrees for different population groups. Because of methodological differences between NMES and the PDEVS, our estimate of pharmacyreported charges could be lower than actual expenditures; hence, our estimate of underreporting is on the conservative side. 14 Our findings indicate that only a small number of respondents can report prescription drug expenditures with any degree of accuracy. Redesigning household drug expenditure surveys may not, by itself, be a viable option for correcting this bias. The PDEVS, like NMES, made an extensive effort to use procedures that would maximize respondent recall. Further redesign efforts are unlikely to be successful if they are dependent on the recall of household respondents. Clearly, program cost calculations that are incorrect to the degree described here have profound implications for the development of public policy measures. Although such estimates will always be subject to uncertainty, the extent of the inaccuracy of household reporting makes such data inappropriate for use in cost projections. While the use of the correction factors developed from the PDEVS can only approximate the extent of bias in current data sets, this methodology is currently the best alternative when making national expenditure estimates from household-reported data. In the long run, however, it is imperative that federal

10 DATAWATCH 155 surveys concerned with drug expenditures integrate social and demographic information from households with use and expenditure data collected directly from pharmacies. The authors thank Harry O'Neii, Ed Keller, and Mark Butler of the Roper Organization for their many contributions to this study. This study was made possible through the financial support of Pfizer, Schering-Plough, Upjohn, and Merck. NOTES 1. Project HOPE, Center for Health Affairs, The Medicare Catastrophic Drug Benefit: An Analysis of the Cost Estimates (Millwood, Va.: Project HOPE, 9 September 1987). 2. One of the primary sources used for cost projections was the 1987 National Medical Expenditure Survey (NMES). As part of the preparation for NMES, researchers spent two years planning the basic design, including refining questionnaires and procedures concerning the survey and its multiple components. In addition, extensive time and effort were devoted to recruiting, training, and supervising interviewers; instrument design; and quality control. The data contractors Westat and the National Opinion Research Center (NORC) are highly regarded in both statistical design and data collection. Nonetheless, the design of NMES, as well as the 1977 and 1980 predecessor surveys, assumes that respondents can accurately recall the use of prescription drugs as well as the total costs associated with each prescription. This assumption has not been made for other components of use and expenditures. Thus, NMES contains verification surveys for household-reported doctor visits, home health care visits, and hospital visits, as well as information concerning insurance coverage. It should be noted that, at the time the survey was designed, there was no way to anticipate the subsequent interest that would shortly evolve in estimating prescription drug expenditures. 3. J. Moeller and N. Mathiowetz, Prescribed Medicines: A Summary of the Use and Expenditures by Medicare Beneficiaries, DHHS Pub. no. (PHS) (Rockville, Md.: National Center for Health Services Research, 1989). 4. Department of Health and Human Services, Report to Congress: Expenses Incurred by Medicare Beneficiaries for Prescription Drugs (Washington, D.C.: DHHS, May 1989), 15, DHHS, Report to Congress: Expenses Incurred by Medicare Beneficiaries for Prescription Drugs, Appendix E (Washington, D.C.: DHHS, May 1989), 16, Congressional Budget Office, Updated Estimates of Medicare's Catastrophic Drug Insurance Program (Washington, D.C.: CBO, 1989). 7. National Center for Health Statistics, Medical Economics Survey Methods Study, Contract no. HRA , The Johns Hopkins Medical Institutions, It should be noted that half of the respondents in the Medical Economics Survey (MES) were given a recall period of one month, while the other half had a recall period of two months. This compares to the two-to-five-month period associated with NMES. Moreover, these estimates are for the population as a whole rather than specifically for the elderly. Studies describing the particular problems of recall among the elderly are summarized in R.M. Groves, Survey Errors and Survey Costs (New York: John Wiley, 1989). 9. DHHS, Report to Congress, Appendix E. 10. Moeller and Mathiowetz, Prescribed Medicines.

11 156 HEALTH AFFAIRS Fall More information on the design of the Prescription Drug Expenditure Verification Survey (PDEVS) can be obtained in a field operations report available from the authors at Project HOPE, Two Wisconsin Circle, Suite 500, Chevy Chase, Maryland From the enumeration of all households with elderly persons within the interviewers' assigned segments, 1,029 persons were determined to be eligible for the study. Seven hundred thirty respondents (71 percent) completed the Round 1 interview. The field period for Round 1 was eight weeks, about half the length of the Round 1 field period on major surveys such as the National Medical Care Expenditure Survey (NMCES) or NMES. Six hundred forty-eight respondents (89 percent) completed the Round 2 interview. Approximately 96 percent of all persons surveyed agreed to complete permission forms authorizing us to contact their pharmacies. Data were obtained from all mentioned pharmacies for 87 percent of respondents to the household survey. Although the response rate on the household component of the PDEVS was somewhat lower than that obtained on NMES, the response rates for the permission forms and for the verification component were somewhat higher than the rates obtained on NMES for those components that were verified (physician visits, inpatient visits, and insurance coverage). 13. It should be noted that only those variables that could be replicated in NMES were used in the correction algorithm we describe. Thus, several variables that in fact contributed to explaining the bias in reporting had to be excluded. Most notably, the number of days a person was confined to bed due to illness performed better than perceived health status as a measure of health but was not available on our version of the NMES file. The overall explanatory power, however, remained consistent across models. Additional information on the development and results of the correction algorithm can be obtained from the authors. 14. Two aspects of the PDEVS design may result in an underestimate of actual expenses. First, the PDEVS was conducted in two waves of interviews covering a six-month reference period, while NMES consisted of four rounds of interviews conducted during the course of a year. Analysis of data from the predecessor of NMES, the 1977 NMCES, indicate that the amount of household reporting may decrease as the survey field period goes on. S.B. Cohen and V.L. Burt, "Data Collection Frequency in the National Medical Care Expenditure Survey," Journal of Economic and Social Measurement 13(1985): Thus, there is some reason to believe accuracy of reporting in a shorter field study such as the PDEVS may be higher than in a study using a one-year reference period as was the case for NMES. A second problem is that it is impossible to verify prescription drug use if the respondent failed to remember the name of a pharmacy where some prescriptions were filled. Barbara Bailar, a consultant to NCHSR and executive director of the American Statistical Association (ASA), evaluated the methodology used in the 1977 Medical Economics Survey. She speculated that respondents failed to accurately recall all of the pharmacies they used. DHHS, Report to Congress, E-10. It should be noted that both of these features of the design would lead to an underestimate of the degree of underreporting that might be found in a household survey. Therefore, the results reported here may somewhat underestimate the degree of bias that may exist in NMES. It should be noted that for purposes of this study, we are using information from pharmacies on drug expenditures as the criterion source. While pharmacy records may be subject to error, a plan such as that proposed under the Medicare Catastrophic Coverage Act would, in fact, rely on those records as the basis for reimbursement of expenses. Thus, in this context, it seems appropriate to regard expenditures as reported in pharmacy records as actual expenditures.