Improving Follow-Up after an Abnormal Pap Smear: Results from a Quasi-Experimental Intervention Study

Transcription

1 JOURNAL OF WOMEN S HEALTH & GENDER-BASED MEDICINE Volume 9, Number 7, 2000 Mary Ann Liebert, Inc. Improving Follow-Up after an Abnormal Pap Smear: Results from a Quasi-Experimental Intervention Study CELIA PATRICIA KAPLAN, Dr.P.H., M.A., 1 ROSHAN BASTANI, Ph.D., 2 THOMAS R. BELIN, Ph.D., 3 ALFRED MARCUS, Ph.D., 4 KIUMARSS NASSERI, Ph.D., 5 and MING-YI HU, M.S. 3 ABSTRACT The success of cervical cancer control programs depends on regular screening with the Pap smear test and prompt and appropriate treatment of early neoplastic lesions. Recognizing the potentially grave consequences of lack of follow-up for abnormal Pap smears, numerous intervention studies have tested the impact of a variety of strategies to increase return for follow-up. The majority of these studies were evaluated under controlled experimental conditions. Despite the encouraging findings of these trials, the next step in the research continuum requires that the effectiveness of these interventions be demonstrated in real world settings before full implementation is initiated. We report the results of an evaluation study assessing the combined effectiveness of three intervention modalities found effective in prior randomized studies: a tracking follow-up protocol, transportation incentives, and financial incentives. This study used a before-after, nonequivalent control group design to assess the impact of a multifaceted intervention that included a computerized tracking protocol with transportation and financial incentives. The study was implemented at two major hospitals, two comprehensive health centers (CHC), and nine public health centers (PHC) under the jurisdiction of the Los Angeles County Department of Health Services. One hospital, one CHC, and the four PHC located in the catchment area of the CHC were selected as experimental sites. The control sites one hospital, one CHC, and five PHC provided usual care. All women with an abnormal Pap smear at the intervention and control sites were included in the study. The study consisted of a 1-year period of baseline data collection (September 1989 August 1990), followed by a 2 1 / 2 -year intervention period (September 1990 February 1993). During the intervention period, the intervention protocol was implemented at the experimental sites, and the control sites provided usual care. Overall, we found that the rates of receipt of follow-up care were consistent with those found in similar studies. In contrast to results obtained in these prior randomized trials, we did not find strong and consistent ev- 1 Division of General Internal Medicine, Department of Medicine, University of California, San Francisco, California. 2 Jonsson Comprehensive Cancer Center and School of Public Health, University of California, Los Angeles, California. 3 Department of Psychiatry and Biobehavioral Sciences and School of Public Health, University of California, Los Angeles, California. 4 AMC Cancer Research Center, Denver, Colorado. 5 California Cancer Registry, Santa Barbara, California. This research was conducted at the Jonsson Comprehensive Cancer Center, University of California, Los Angeles, through Public Health Service grant No. PO1 CA

2 780 KAPLAN ET AL. idence for intervention effects. Significant findings emerged only at the CHC and hospital levels and only for selected years. Results underscore the importance of testing interventions in real world conditions before large-scale implementation is initiated. In addition, this study highlights the challenge of detecting intervention effects in large-scale studies because of the greater measurement difficulties in field studies as compared with controlled experiments. INTRODUCTION DURING THE LAST THREE DECADES, the United States has experienced a sharp decrease in cervical cancer incidence and mortality. 1 Despite this documented reduction, the American Cancer Society estimates that in the year 2000, approximately 16,000 new cases of invasive cervical carcinoma will be identified and about 5000 women will die as a result. 2 Furthermore, the burden of this disease is not evenly distributed in the United States population. It disproportionately strikes low-income, elderly, Latina, and African American women The success of cervical cancer control programs depends on regular screening with the Pap smear test, particularly among high-risk groups. However, a comprehensive cervical cancer screening program by itself is not sufficient in reducing cervical cancer incidence and mortality. Prompt and appropriate treatment of early neoplastic lesions identified through screening is also a critical component, ensuring that women receive timely, state-of-the-art medical care The importance of follow-up of early lesions should be emphasized, as delays in treatment may result in increased risk of developing more severe disease. 16 Furthermore, women who are initially identified by screening as having low-grade abnormalities could be found to have more advanced lesions when more extensive histological examinations are performed. 17 Despite the importance of follow-up procedures, evidence indicates that 20% 50% of women with cervical abnormalities do not return for post-screening care Such loss to followup may be a significant factor contributing to the large numbers of women still diagnosed with, and dying from, cervical cancer. Recognizing the potentially grave consequences of lack of follow-up for abnormal Pap smears, numerous controlled intervention studies have tested the impact of a variety of strategies to increase return for follow-up. These studies have targeted primarily high-risk populations. Several intervention modalities have been found to be effective. Educational pamphlets alone 13,31 35 or in combination with a provider intervention, 24 telephone counseling, 21 intensive case management, 25 transportation incentives, financial incentives, 18 and a tracking follow-up protocol 36 have been found to improve return for follow-up care. These studies were conducted as randomized trials in which implementation of interventions was closely monitored and greater resources were available to conduct research. However, results obtained from these controlled randomized trials may not be generalizable to the actual settings where interventions are implemented. To ascertain the ultimate effect of any intervention, it needs to be implemented and evaluated in multiple and disparate settings. 37,38 The present study attempted to implement and evaluate three intervention modalities found to have positive effects in prior randomized studies: a tracking follow-up protocol that includes a personal letter and informational brochures, transportation incentives, and financial incentives. 18,36 We chose a before-after, nonequivalent control group design to assess its effectiveness. The choice of this design was dictated largely by practical considerations, such as resource limitations and logistical constraints posed by the healthcare system. In particular, given the characteristics of the healthcare system where the study was implemented, a sufficient number of sites was not available for randomization to intervention and control conditions, nor was it possible, because of the structure of the healthcare system, to randomize the intervention to only a portion of the clients attending a particular site. This study was conducted as part of a program project, the Cancer Prevention Research Unit of Los Angeles (CPRU), aimed at increasing cervical and breast cancer screening and Pap smear follow-up at healthcare facilities of the Los Angeles County Department of Health Services (LACDHS). We report the final results of this study.

3 FOLLOW-UP OF CERVICAL ABNORMALITIES 781 MATERIALS AND METHODS Research design This study used a before-after, nonequivalent control group design. 39 The study was implemented at two major hospitals, two comprehensive health centers (CHC), and nine public health centers (PHC) under the jurisdiction of the LACDHS. One hospital, one CHC, and the four PHC located in the catchment area of a CHC were selected as experimental sites. One hospital, one CHC, and five PHC in the control sites provided usual care. All women with an abnormal Pap smear at the intervention and control sites were included in the study. The study consisted of a 1-year period of baseline data collection (September 1989 August 1990), followed by a 2 1 / 2 -year intervention period (September 1990 February 1993). During the intervention period, the intervention protocol was implemented at the experimental sites, and the control sites provided usual care. There was, however, one noteworthy exception. At the intervention and control hospitals, implementation of the CPRU project overlapped for 2 years with another randomized intervention study targeting women with abnormal Pap results. This overlap affected only those patients attending selected clinics at the two hospital sites (gynecology and emergency room at the CPRU control site and gynecology and family clinic at the CPRU experimental site). Therefore, in these hospital clinics, the usual care control group (randomly assigned) of the other intervention study was the only source of baseline information for our CPRU study. In these clinics, the CPRU intervention was initiated after the other study was fully completed, which amounted to treating the first 2 years in these clinics rather than only the first year as baseline period. This change in the intervention schedule at these two sites was implemented to assure that the effects of the previous study s intervention would not contaminate the results of this CPRU study. Computerized reports and log books provided by the cytology laboratories under contract to the LACDHS were used to identify patients with abnormal Pap smears. For the purposes of this study, abnormal Pap smears were defined as those with low-grade squamous intraepithelial lesions (SIL) or more severe abnormalities. Patients with human papillomavirus (HPV) or koilocytotic atypical changes without SIL were not included in the study, as some of the sites referred these cases to outside clinics, where it was not feasible to track follow-up care. The analyses excluded women who had had a prior abnormal Pap smear during the preceding 12 months or who had been previously diagnosed with cervical cancer, as return for follow-up could be affected by a recent prior abnormal cytological diagnosis. The study also excluded referrals from non-lacdhs facilities. Over the 3 1 / 2 years of study enrollment, a total of 4488 women identified with an abnormal Pap smear were deemed eligible for this study. These included 2099 subjects at the hospital level (834 intervention, 1265 control), 1661 at the CHC level (1240 intervention, 421 control), and 728 at the PHC level (382 intervention, 346 control). Overall, we included 2456 subjects in the intervention group and 2032 in the control group. Description of the intervention Lack of adherence to follow-up recommendations is affected by a multiplicity of interrelated factors that include the characteristics of the healthcare system structure, barriers to access, and psychosocial factors. Given the complexity of the problem, the development and implementation of the intervention reflects several complementary theoretical perspectives: Dietrich s office system approach, 40,41 Andersen s framework of healthcare utilization, 42 and the health belief model. 43 Domains derived from these frameworks were operationalized and included in the three components of the intervention: a centralized computer tracking follow-up, transportation incentives, and financial incentives. One likely critical factor contributing to lack of adherence to follow-up is the structure and functioning of the healthcare system. For example, lack of a systematic and organized approach to reach patients affects the ability of providers to contact patients in an effective and timely manner. An office system approach 40,41 addresses this issue by implementing specific routines (e.g., a systematic follow-up protocol), providing tools (e.g., a computerized system), and assigning specific roles for office staff and physicians that consistently encourage the follow-up of abnormalities. Irrespective of the healthcare system organization, there are many factors that may affect a

4 782 woman s decision to obtain the necessary followup care. Andersen s model of healthcare utilization 42 illustrates that health-seeking behavior is dependent on the predisposition of an individual to use services, the need to receive healthcare, and the ability to secure these services. This model emphasizes the importance of identifying and overcoming any concrete barriers the patient may have to obtaining the required health services. In this study, implementation of both financial and transportation incentives addressed two potential barriers or enabling factors to action. Psychosocial factors may also impede adherence to follow-up. The health belief model, 43 which addresses these components, provided us with a useful framework for the intervention strategy and materials. A patient s perceptions of her health diagnosis, as well as beliefs about the effectiveness of treatment, are central to how she behaves. To affect these perceptions and beliefs in a positive manner, the intervention included letters, telephone calls, and brochures all designed to address the patient in a personal manner. The provider/patient communications served as direct cues to action to promote adherence to follow-up by clarifying what actions need to be taken; when, where, and how to proceed; and what positive effects could be expected if followthrough occurred. Specifically, written information was offered to each patient about her personal susceptibility to cervical cancer and the efficacy of the tests and treatments. KAPLAN ET AL. Centralized computerized tracking follow-up. To coordinate delivery of the intervention, a CPRU office was established for the intervention group at both the CHC and hospital. Each office was staffed with one or two follow-up coordinators responsible for identifying, scheduling, and tracking all women with an abnormal Pap smear. The staff at the CPRU offices performed all follow-up duties at the intervention sites, replacing any existing follow-up activity. A special computerized program was designed to initiate and monitor all follow-up activities. All coordinators hired for this study were bilingual in Spanish and English and had extensive experience in women s health. Data from women identified with an abnormal Pap smear were entered into the computerized database, including patient demographic information, address, telephone number, and degree of abnormality. After identification of an abnormal screening, a letter was sent to notify the patient of the need for follow-up care and the date and time of her follow-up appointment. In addition, a reminder letter was mailed to the patient 1 week before the scheduled appointment. The letter, addressed directly to the patient, was written in the patient s language of preference (obtained from the patient s chart) and signed by the chief physician responsible for the abnormal cytology clinic. The content of the letter was tailored to the type and seriousness of the Pap smear abnormality and included a brief description of the results, an indication of what further tests might be required, instructions advising the woman to return for follow-up at the appointed time, and the level of urgency of the recommendation. The letter also advised the patient to read an enclosed pamphlet about Pap smears. This accompanying pamphlet, titled Your Pap Smear: Understanding the Results, explained a Pap smear and the meaning of the results and described tests and treatments for women diagnosed with a cervical abnormality. All text was in both English and Spanish. If a letter was returned as undeliverable from the post office, the CPRU site coordinator (at the follow-up office) verified the address using the site s computerized patient registration system. If a new address was found on the computer, the letter was resent to the patient. In addition, if the patient did not keep her appointment or if the letters were undeliverable, a telephone protocol was initiated, with four telephone call attempts made to reach the patient. If the patient was contacted, she was informed of the follow-up appointment, and, if necessary, her address was updated. All contacts and attempts to contact the patient were systematically entered into the computerized follow-up system. In those cases where carcinoma in situ or invasive cancer was suspected and the patient did not return for follow-up, the LACDHS policy dictated that home visits by visiting nurses be attempted to notify patients. Transportation incentives. All patients, irrespective of level of seriousness of the abnormality, received transportation incentives in the form of free bus passes. These passes, mailed with the initial letter, included a brief note indicating that the passes were sent to assist patients in returning for follow-up care and could be used within the Los Angeles County transportation system. Enough passes were sent to allow the patient two roundtrip fares.

5 FOLLOW-UP OF CERVICAL ABNORMALITIES 783 Financial incentives. For patients with highgrade SIL or a more severe degree of abnormality, a financial incentive in the form of a voucher was enclosed with the letter. This voucher could be used as partial payment for follow-up visits related to the abnormal Pap smear, supplementing the low-cost healthcare plan that already existed at all LACDHS sites. Within this financial plan, patients without public or private insurance who did not qualify for a payment waiver were required to pay a $40 visit fee within 7 days of the visit. The voucher was designed to offset this expense, covering $15 of the cost of the visit to the abnormal cytology clinic. The voucher was accompanied by a pamphlet explaining who could use the voucher, what services it covered, and how to use it at each facility. As with all other educational and informational materials used in the study, the vouchers and pamphlets were written in both Spanish and English. Follow-up at control sites. Sites assigned to the control group continued to provide standard LACDHS follow-up notification regarding abnormal Pap smears. At each control site, follow-up remained the responsibility of the clinic clerk or nurse assigned to follow-up cervical abnormalities. These follow-up duties were in addition to their other clinic responsibilities. In contrast, the follow-up coordinators at the intervention sites had only one responsibility, which was to follow-up patients with abnormalities. Notification procedures varied at different control facilities. In general, the procedures entailed mailing a form letter requesting the patient to return for follow-up care at a certain location and time. These letters, typically, did not inform patients of their test results, nor did they provide any incentives to return. No protocol was in place at any of the control sites to follow-up patients who did not return, and no protocols were in place to locate patients with wrong addresses or telephone numbers. One exception to this lack of follow-up procedure at the control sites was the previously established protocol for patients with carcinoma in situ or invasive cancer who did not return for follow-up visits. As in the intervention condition with these most serious of cases, home visits by visiting nurses were mandated. Data collection The main outcome of interest was whether the patient received complete follow-up care for an abnormal Pap smear during the 9-month period following identification of the cervical abnormality. Based on prior research and examination of clinic data, a 9-month follow-up period was considered sufficient time for most patients to complete relevant follow-up tests and treatments. Data were collected from three sources: patient medical records, the LACDHS computerized patient information system, and computerized files obtained from the cytology laboratory under LACDHS contract. Patient medical records. The patient medical records served as the primary source of information on follow-up care. Chart audits were conducted at all clinics and facilities where the patient was referred for medical care to ascertain (1) demographic information, including race/ethnicity, age at the time of the abnormality, country of birth, and marital status, (2) medical information, such as degree of abnormality of the Pap smear, tests and procedures received, dates of tests and procedures received, patient s pregnancy status at the time of the abnormal Pap smear, and history of previous cervical abnormalities, and (3) clinics (other than abnormal cytology clinics) visited during the 9-month followup period. Finally, the chart reviewers rated each case as to whether the patient received full, partial, or no follow-up care. Chart audits were conducted by eight medical record abstractors who received training at the clinic sites under the direct supervision of the data collection field coordinator. All forms were reviewed for completion by a trained research assistant who completed missing demographic information using the LACDHS patient computerized information system and the cytology computerized data files. Patient computerized data system. If demographic data in the medical records were missing or incomplete or medical records could not be retrieved, information on date of birth, marital status, ethnicity, and place of birth was supplemented by the LACDHS patient computerized data system. Cytology computerized data files. When it was not possible to locate the chart, supplemental information was obtained from the cytology computerized data files. Using patient file numbers and names for matching, the cytology data files were

6 used to determine if the woman had received at least one additional pap smear following identification of an abnormality. Outcome measures Receipt of follow-up care was measured by the chart reviewers ratings of the extent to which recommended examinations and treatments were obtained during the 9-month follow-up period. Based on the examinations and treatments received and the health providers notes and recommendations, a three-point scale was used to assess whether the patient received no care, partial care, or complete care. Thus, if the medical record indicated that a patient needed to return for follow-up care but did not return for any care, the medical record abstractor was instructed to rate the patient as having received no care. On the other hand, if all examinations and procedures recommended during the study period were performed, the woman was rated as having received complete care. Women who received some but not all of the procedures recommended were classified as having received partial care. For the purposes of this study, no care and partial care were combined to indicate women who received incomplete care. The dichotomized rating (complete care vs. incomplete care) served as the main dependent variable in this study. Within the LACDHS system, all women identified with cervical abnormalities at the PHC level are routinely referred to a higher level facility (usually a CHC), whereas in most cases at the CHC and hospital levels, women receive followup care at the same facility. Women with very severe lesions (carcinoma in situ and higher degrees of abnormality) at CHC and PHC are referred to the hospital. Thus, it was necessary to develop an algorithm that included the receipt of care ratings from each level where the patient could have received care to determine the final outcome. Because women identified with an abnormality at a PHC were routinely referred to a CHC, the rating reported at the CHC level determined the final disposition. Women identified at a CHC and not referred to a hospital were classified according to the disposition obtained from the CHC medical chart. For patients at the CHC or PHC who were referred to the hospital, a combined measure of receipt of care was created. This measure took into consideration whether the patient completed the recommended tests and treatments at both the CHC and hospital levels. 784 The final outcome for women whose patient records were not available was derived from the cytology computerized data files. After a matching process that took into consideration patient file number, last and first name, and possible misspellings of name, a final outcome was assigned based on the cytology computerized data files. If a Pap smear was performed during the 9-month follow-up period, the woman was considered as having received at least some follow-up care. As it was not possible to determine if this group of women received complete care, they were classified as receiving partial care. Statistical analysis KAPLAN ET AL. Because of concerns about the comparability of sites representing different levels of available health services, separate analyses were carried out on data from hospitals, CHC, and PHC. Descriptive summaries of characteristics of the study population are supplemented with intervention-vs.-control comparisons using chi-square test to assess possible differences in case mix. A multiple logistic regression model was used to predict whether a given patient received complete vs. incomplete follow-up care during the 9- month period after the initial Pap smear visit. In the analysis, subjects who received either partial care or no care were classified as having received incomplete follow-up care. The logistic regression model controlled for basic sample characteristics, including age (continuously scaled), ethnicity (Latina vs. non-latina), marital status (married, not married, or unknown marital status), degree of abnormality (high-grade SIL vs. low-grade SIL), previous history of abnormality (yes vs. no), and pregnancy status (pregnant vs. not pregnant). The model also included binary indicator variables reflecting intervention vs. control group status and study year (baseline, year 02, year 03, or year 04) from which each observation arose. With the control group at baseline serving as the reference group, seven indicator variables sufficed to represent the eight possible combinations of these two variables. To assess whether changes over time in the intervention group were significantly different from changes in the control group, controlling for case-mix covariates, contrasts were constructed between logistic regression coefficients. For example, a possible difference between the change in receipt of complete care from baseline to year 02 in the intervention site vs. the corresponding

7 FOLLOW-UP OF CERVICAL ABNORMALITIES 785 change at the control site was captured by calculating [((l 2 ) 2 (l 0 )) 2 (C 2 )], where (l 2 ) is the logistic regression coefficient for the indicator of the intervention site in year 02, (l 0 ) is the coefficient for the intervention group at baseline, and (C 2 ) is the coefficient for the control group in year 02. As the control group at baseline is the reference group, there is no coefficient for the control group at baseline. A standard error for each such contrast was obtained by summing the appropriate variance and covariance terms from the covariance matrix of the fitted logistic regression coefficients and then taking the square root. Odds ratios (OR) and 95% confidence intervals (CI) are derived from the contrasts. Two-sided p values were used to judge statistical significance. Description of sample RESULTS Table 1 presents the demographic characteristics of women with cervical abnormalities at each healthcare level for the intervention and control sites for all study years. Combining data across all years of the study, patients at the PHC were younger than those at the hospital and CHC levels. More than 35% of the sample at each level reported being married. The CHC and PHC levels had a higher proportion of Hispanics than the hospital level. With respect to severity of cervical abnormality and prior history of cervical abnormality, the data indicate no major differences among the three levels. Finally, there was a higher proportion of pregnant women (at time of the initial Pap smear) at the CHC and PHC than at the hospital level, reflecting the service mix provided at each of these sites. In comparing patient characteristics between the control and the intervention conditions, significant differences are found at each level of care, with particularly large differences at the hospital level. Clients at the hospital control site tended to be younger, had Pap smears with a lower degree of abnormality, had a higher likelihood of previous cervical abnormalities, and were more likely to be pregnant than women in the intervention hospital. There were significantly more Latinas within the intervention group compared with the control sample at the hospital level. At the CHC level, women in the control group were more likely to be pregnant and of Latina origin than those in the intervention group. At the PHC level, women in the control group were more likely to be young, married, and pregnant than those in the intervention group. Subject characteristics were entered as control variables in all multivariate analyses assessing the impact of the intervention. Return for follow-up care: Descriptive results Table 2 shows the proportion of women who received complete care in the 9-month follow-up period by level of care, intervention and control group, separately for each study year. These data indicate that overall, irrespective of level of care TABLE 1. CHARACTERISTICS OF WOMEN WITH ABNORMAL PAP SMEARS Hospital CHC PHC I a C I C I C (n 5 834) (n ) (n ) (n 5 421) (n 5 382) (n 5 346) (%) (%) (%) (%) (%) (%) Age (years) *** ** Ever married ** Hispanic *** *** High grade *** Previous abnormal smear * Pregnant *** *** *** No. of sites a I, intervention group; C, control group. *p # 0.05; **p # 0.01; ***p # of I-C comparisons.

8 786 KAPLAN ET AL. TABLE 2. PERCENTAGE OF WOMEN RECEIVING COMPLETE CARE Hospital CHC PHC Total I a C I C I C I C (n 5 834) (n ) (n ) (n 5 421) (n 5 382) (n 5 346) (n ) (n ) Baseline Year 01 (227) (491) (354) (117) (100) (97) (681) (705) Year * (134) (347) (340) (138) (102) (108) (576) (593) Year * (336) (340) (360) (120) (122) (97) (818) (557) Year * (137) (87) (186) (46) (58) (44) (381) (177) Years *** (607) (774) (886) (304) (282) (249) (1775) (1327) a I, intervention group; C, control group. *p # 0.05; ***p # of I-C comparisons. and intervention or control assignment,,50% of the women received complete care. At the hospital level, complete care was received by 36.6% of the intervention group at baseline and by 40.9% of the intervention group across the intervention years. Compared with the baseline year, the rate of complete care was lower in year 02 and higher in years 03 and 04. However, neither the overall comparison between baseline and intervention years combined nor the separate comparisons between baseline and the individual follow-up years showed significant differences. In the control group, rates of complete care remained essentially constant over time (34.6% at baseline vs. 35.5% for post-baseline years combined), with a similar pattern of a lower rate in year 02 and higher rates in years 03 and 04 but with no significant differences. The CHC level reveals a different pattern of follow-up care from the hospital level. Comparison of the baseline period and the combined intervention years indicate that both the intervention and control groups experienced a decrease in the receipt of complete care (44.6% to 39.2% in the intervention group and 48.7% to 40.1% in the control group). No time trend is apparent in either group. However, a significantly higher proportion of women in the control group than in the intervention group received complete care in year 03 of the study. At the PHC level, comparison of baseline vs. all intervention years combined indicates that the intervention group experienced an increase in the receipt of complete care (34.0% to 42.6%). The control group did not show a substantial increase over the study period (28.9% compared with 30.5%). Examination of time trends indicates a steady increase through year 03 in both groups, followed by a drop in year 04. This decrease is more noticeable in the control group and actually fell below the baseline rate. No significant differences were found between intervention and control groups. We also analyzed the data collapsed across type of facility. Examination of all facilities combined indicates an intervention effect in year 02 and year 04 of the study. Logistic regression analysis to assess intervention effectiveness Logistic regression analyses were used to compare the intervention vs. control conditions on receipt of complete follow-up, controlling for differences in case mix between sites (Table 3). At the hospital level, the contrast coefficient for baseline vs. all intervention years combined indicates that there was no overall significant intervention effect. Comparisons between baseline and individual follow-up years show that only the comparison between baseline and year 03 is statistically significant (OR 1.65), suggesting an intervention group advantage in return to follow-up care for that year. At the CHC level, no overall intervention effect was identified. A borderline trend in favor of the intervention group is seen in the comparison between baseline and year 04. No overall significant intervention effect is observed at the PHC level, although all the point estimates comparing inter-

9 TABLE 3. CONTRASTS COMPARING COMPLETE FOLLOW-UP RATES BETWEEN INTERVENTION AND CONTROL GROUPS a Hospital CHC PHC Contrasts Contrast (SE) OR (95% CI) Contrast (SE) OR (95% CI) Contrast SE OR (95% CI) Baseline vs. year (0.28) 1.24 (0.71, 2.16) 0.40 (0.31) 1.50 (0.82, 272) 0.10 (0.43) 1.10 (0.47, 2.57) Baseline vs. year * (0.24) 1.65 (1.03, 2.64) (0.31) 0.77 (0.42, 1.42) 0.06 (0.43) 1.06 (0.46, 2.44) Baseline vs. year (0.34) 1.34 (0.69, 2.61) 0.68 (0.42) 1.97 (0.86, 4.50) 0.32 (0.56) 1.38 (0.46, 4.12) Change fron year 02 to (1.33) 1.33 (0.77, 2.28) (0.31) 0.52 (0.28, 0.94) (0.41) 0.96 (0.43, 2.13) Change from year 03 to (0.82) 0.82 (0.42, 1.57) 0.93** (0.42) 2.54 (1.11, 5.83) 0.27 (0.54) 1.31 (0.45, 3.77) Baseline vs. all intervention years 1.01 (2.74) 2.74 (0.75, 10.00) 0.82 (0.81) 2.28 (0.47, 11.13) 0.48 (1.14) 1.61 (0.17, 14.90) a Based on results from logistic regression, controlling for age, race/ethnicity, grade of abnormality, pregnancy status, previous abnormality, and marital status. *p, 0.05; **p

10 vention and control groups between baseline and subsequent years are in the positive direction. DISCUSSION The main goal of this study was to assess the effects of a multifaceted intervention aimed at increasing follow-up care for women with abnormal Pap smears at public health facilities. The intervention had three main components: central computerized tracking and follow-up in the form of personalized letters and informational brochures, transportation incentives, and financial incentives. A number of previous studies 18,36 in similar settings have shown that these interventions were effective in increasing receipt of follow-up care among low-income women with abnormal Pap smears. Overall, we found that the rates of receipt of follow-up care were consistent with those found in similar studies addressing the follow-up of women with abnormal Pap smears, 12,19,29 although they varied by level of care and year of implementation. In contrast to results obtained in these prior randomized trials, we did not find strong evidence for intervention effects. Significant findings emerged only at the CHC and hospital levels and only for selected years. In particular, the significant intervention effects were confined to the later years of the study, possibly because of increased efficiency by the facilities in the experimental groups in delivering the intervention. Although we found a positive trend at the PHC level, this trend did not reach statistical significance, probably in part because of the smaller sample size available at this level. Although the study used a quasi-experimental design, we were able to control statistically for differences in study population. Additionally, the analyses took into consideration possible structural and functional differences among the healthcare sites by statistically controlling baseline rates at both intervention and comparison facilities. These rates are a proxy for variations in staffing patterns, staff capabilities, space resources, and interactions among these factors that affect overall medical care and, therefore, need to be controlled in statistical analysis comparing intervention and comparison sites. Despite these statistical controls, it is still possible that intervention effects were underestimated. Lack of overall statistical significance 788 KAPLAN ET AL. could be the result of several factors, including outside events that influenced the functioning of LACDHS and contamination effects difficult to control in a quasi-experimental design. One can hypothesize that outside events may have operated to counteract implementation efforts, as was the case in this study. LACDHS experienced a significant decrease in funding that threatened the closure of county CHC, PHC, and hospitals during the course of this study. As a result of this wave of funding cuts, waiting times for appointments at colposcopy clinics increased over time and may have negatively affected return rates. This problem may have been more acute in facilities already functioning at full capacity, which was the case at our intervention hospital and CHC. In fact, the intervention CHC experienced a decline in year 02 of the study not experienced by the control facility. In times of severe funding reductions, it may have been more difficult for these facilities to accommodate large numbers of patients. In this study, the facilities chosen as intervention sites were the facilities with the highest patient volume. For example, the CHC assigned to the intervention condition had more than double the number of abnormal Pap smears as the control CHC during the same period. Although the intervention may not have increased the rate of return for receipt of complete care, it may have allowed the facilities to maintain preintervention levels of care that otherwise would have decreased. The lack of consistent significant effects may also be due to contamination problems. Representatives of facilities assigned to the control condition were aware of the elements included in the intervention and participated in study-steering committee meetings. This knowledge and participation may have promoted greater efforts in improving return rates at the control facilities. It is also feasible that the lack of intervention effects at the hospital level may be due to a spillover effect from the prior study 36 that could have altered the baseline information. Although this is theoretically possible, the prior study found significant effects in the intervention condition that suggest that the intervention did not affect the control condition. In addition, return rates (in the prior study) in the control condition of the randomized trial are equivalent to those found in this study. Although the site coordinators were stationed at the hospital sites during the randomized trial, all follow-up procedures were imple-

11 FOLLOW-UP OF CERVICAL ABNORMALITIES 789 mented from the Jonsson Comprehensive Cancer Center and were independent of the hospitals, which continued to deliver usual care throughout the project. In summary, the study represents a timely and valuable inquiry into the effectiveness of a cancer control intervention. Despite positive results found in prior controlled trials, no significant effects were detected in the current study. Significant effects were identified at certain facilities only during the later years of intervention implementation. These results underscore the importance of testing interventions in actual settings before large-scale implementation is initiated. In addition, this study highlights the challenge of detecting intervention effects in large-scale studies because of the greater measurement difficulties in field studies as compared with controlled experiments. Thus, we encourage further population-based research to ascertain the effectiveness of interventions outside of tightly controlled research settings. ACKNOWLEDGMENTS We thank our colleagues and collaborators from the Los Angeles County Department of Health Services: Caswell Evans, D.D.S., Sol Bernstein, M.D., Ellen Alkon, M.D., M.P.H., Carol Henneman, M.D., and Mary Abbott, M.D. We also gratefully acknowledge the contribution of Rosa Trejo, Shannon Frattaroli, Pamela Brown, and Howard Martin for their assistance with data collection and Frances Bernabe, Gloria Chavez, Gabriela Salas, and Laura Sarmiento for project implementation. We thank the Cancer Screening Service, Inc., for their cooperation and help in conducting this study. REFERENCES 1. Dewar MA, Hall K, Perchalski J. Cervical cancer screening: Past success and future challenge. Primary Care 1992;19: American Cancer Society. Cancer facts and figures Atlanta, GA: ACS. 3. Cancer incidence among NYC Hispanics NY State J Med 1990;Jan 90(1): Savitz DA. Changes in Spanish surname cancer rates relative to other whites, Denver area, to Am J Public Health 1986;76: Rosenwaike I. Cancer mortality among Mexican immigrants in the United States. Public Health Rep 1988;103: Mack TM, Walker A, Mack W, Bernstein L. Cancer in Hispanics in Los Angeles County. Natl Cancer Ins Monogr 1985;69: Peters RK, Thomas D, Hagan DG, Mack TM, Henderson BE. Risk factors for invasive cervical cancer among Latinas and non-latinas in Los Angeles County. J Natl Cancer Inst 1986;77: Becker RM, Wheeler CM, Key CR, Samet JM. Cervical cancer incidence and mortality in New Mexico s Hispanics, American Indians, and non-hispanic whites. West J Med 1992;156: Mallin K, Anderson K. Cancer mortality in Illinois Mexican and Puerto Rican immigrants, Int J Cancer 1988;41: Centers for Disease Control. Black-white differences in cervical cancer mortality United States, Public Health Rev 1989/1990;17: Garfinkel L, Poindexter C, Silverger E. Cancer in black Americans. Cancer 1980;30: Cella D, Orav E, Kornblith A, et al. Socioeconomic status and cancer survival. J Clin Oncol 1991;9: Koss LG. The Papanicolaou test for cervical cancer detection. JAMA 1989;261: Howard J. Avoidable mortality from cervical cancer: Exploring the concept. Soc Sci Med 1987;24: Miller KE, Losh DP, Folley A. Evaluation and followup of abnormal pap smears. Am Fam Phys 1992;45: Creasman WT, Clarke-Pearson DL. Abnormal cervical cytology: Spotting it, treating it. Contemp Obstet Gynecol 1986;21: Bolger BS, Lewis BV. A prospective study of colposcopy in women with mild dyskaryosis or koilocytosis. Br J Obstet Gynaecol 1988;95: Marcus AC, Crane LA, Kaplan CP, et al. Improving adherence to screening follow-up among women with abnormal Pap smears. Med Care 1992;30: Kaplan CP, Bastani R, Marcus A, Breslow L, Nasseri K, Chen L. Low income women with cervical abnormalities: Individual and system factors affecting follow-up. J Women s Health 1995;4: Schwartz PJ, Fasal E, Simmons ME. 12-county program: Screening of 34,318 women for cervical cancer in California, Public Health Rep 1981;96: Lerman C, Hanjani P, Caputo C, et al. Telephone counseling improves adherence to colposcopy among lower-income minority women. J Clin Oncol 1992;10: Lane DS. Compliance with referrals from a cancerscreening project. J Fam Pract 1983;17: Lacey L, Whitfield J, DeWhite W, et al. Referral adherence in an inner city breast and cervical cancer screening program. Cancer 1993;72: Michielutte R, Diseker RA, Young LD, May WJ. Noncompliance in screening follow-up among family planning clinic patients with cervical dysplasia. Prev Med 1985;14:248.

12 Frisch LE. Effectiveness of case management protocol in improving follow-up and referral of Papanicolaou smears indicating cervical intraepithelial neoplasia. J Am Coll Health 1986;35: Cartwright PS, Reed G. No-show behavior in a county hospital colposcopy clinic. Am J Gynecol Health 1990; 4: Soutter WP, Wisdom S, Brough AK, et al. Should patients with mild atypia in a cervical smear be referred for colposcopy? Brit J Obstet Gyn 1986;93: Robertson JH, Woodend BE, Crozier EH, et al. Risk of cervical cancer associated with mild dyskryosis. Br Med J 1988;297: Spitzer M, Krumholz BA, Chernys AE, et al. Cooperative utility of repeat Papanicolaou smears, cervicography, and colposcopy in the evaluation of atypical Papanicolaou smear. Obstet Gynecol 1987;69: Richart RM, Townsend DE, Crisp W, et al. An analysis of long-term follow-up results in patients with cervical intraepithelial neoplasms treated by cryotherapy. Am J Obstet Gynecol 1980;137: Laedtke TW, Dignan M. Compliance with therapy for cervical dysplasia among women of low socioeconomic status. South Med J 1992;85: Paskett ED, Carter WB, Chu J, White E. Compliance behavior in women with abnormal Pap smears. Med Care 1990;28: Paskett ED, White E, Carter WB, Chu J. Improving follow-up after an abnormal Pap smear: A randomized controlled trial. Prev Med 1990;19: Stewart DE, Lickrish GM, Sierra S, Parking H. The effect of educational brochures on knowledge and emotional distress in women with abnormal Papanicolaou smears. Obstet Gynecol 1993;81: Stewart DE, Buchegger PM, Lickrish GM, Sierra S. The effect of educational brochures on follow-up compliance in women with abnormal papanicolaou smears. Obstet Gynecol 1994;83: Marcus A, Kaplan CP, Crane LA, Berek JS, Bernstein KAPLAN ET AL. G, Gunning JE. Reducing loss to follow-up among women with abnormal Pap smears: Results from a randomized trial testing an intensive follow-up protocol and economic incentives. Med Care 1998;36: Greenwald P, Cullen JW, McKenna JW. Cancer prevention and control: From research through applications. J Natl Cancer Inst 1987;79: Paskett ED, Tatum CM, D Agostino R Jr, et al. Community-based interventions to improve breast and cervical cancer screening: Results of the Forsyth County Cancer Screening (FoCas) Project. Cancer Epidemiol Biomarkers Prev 1999;8: Campbell D, Stanley J. Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin, Dietrich AJ, Sox CH, Tosteson TD, Woodruff CB. Durability of improved physician early detection of cancer after conclusion of intervention support. Cancer Epidemiol Biomarkers Prev 1994;3: Dietrich AJ, Woodruff CB, Carney PA. Changing office routines to enhance preventive care. Arch Fam Med 1994;3: Andersen R. A behavioral model of families use of health services. Chicago: University of Chicago, Center for Health Administration Studies. 43. Becker MH, Haefner DP, Kasl DP, Kirscht, JP, Maiman LA, Rosestock IM. Selected psychosocial models and correlates of individual health-related behaviors. Med Care 1977;15(Suppl):27. Address reprint requests to: Celia Patricia Kaplan, Dr.P.H., M.A. University of California, San Francisco Division of General Internal Medicine Department of Medicine 3333 California Avenue, Room 335-G San Francisco, CA