Costs, Effectiveness, and Workload Impact of Management Strategies for Women With an Adnexal Mass

Transcription

1 JNCI J Natl Cancer Inst (2015) 107(1): dju322 doi: /jnci/dju322 First published online December 16, 2014 Article Costs, Effectiveness, and Workload Impact of Management Strategies for Women With an Adnexal Mass Laura J. Havrilesky, Michaela Dinan, Gregory P. Sfakianos, Lesley H. Curtis, Jason C. Barnett, Toon Van Gorp, Evan R. Myers Affiliations of authors: Division of Gynecologic Oncology (LJH), and Department of Obstetrics and Gynecology (LJH, ERM), Department of Medicine (MD, LHC), Duke University Medical Center, Durham, NC; Duke Cancer Institute, Durham, NC (LJH, ERM); St. Francis Hospital, Columbus, GA (GPS); Division of Gynecologic Oncology, Department of Obstetrics and Gynecology, San Antonio Military Medical Center, Fort Sam Houston, TX (JCB); Division of Gynaecological Oncology, Department of Obstetrics and Gynaecology, Universitaire Ziekenhuizen Leuven, Katholieke Universiteit Leuven, Leuven, Belgium (TVG); Division of Gynaecological Oncology, Department of Obstetrics and Gynaecology, MUMC., GROW School for Oncology and Developmental Biology, Maastricht, the Netherlands (TVG). Correspondence to: Laura J. Havrilesky, MD, MHSc, Box 3079 DUMC, Durham, NC ( havri001@mc.duke.edu). Abstract Background: We compared the estimated clinical outcomes, costs, and physician workload resulting from available strategies for deciding which women with an adnexal mass should be referred to a gynecologic oncologist. Methods: We used a microsimulation model to compare five referral strategies: 1) American Congress of Obstetricians and Gynecologists (ACOG) guidelines, 2) Multivariate Index Assay (MIA) algorithm, 3) Risk of Malignancy Algorithm (ROMA), 4) CA125 alone with lowered cutoff values to prioritize test sensitivity over specificity, 5) referral of all women (Refer All). Test characteristics and relative survival were obtained from the literature and data from a biomarker validation study. Medical costs were estimated using Medicare reimbursements. Travel costs were estimated using discharge data from Surveillance, Epidemiology and End Results Medicare and State Inpatient Databases. Analyses were performed separately for pre- and postmenopausal women ( subjects in each), repeated times. Results: Refer All was cost-effective compared with less expensive strategies in both postmenopausal (incremental costeffectiveness ratio [ICER] $9423/year of life saved (LYS) compared with CA125) and premenopausal women (ICER $10 644/ YLS compared with CA125), but would result in an additional 73 cases/year/subspecialist. MIA was more expensive and less effective than Refer All in pre- and postmenopausal women. If Refer All is not a viable option, CA125 is an optimal strategy in postmenopausal women. Conclusions: Referral of all women to a subspecialist is an efficient strategy for managing women with adnexal masses requiring surgery, assuming sufficient capacity for additional surgical volume. If a test-based triage strategy is needed, CA125 with lowered cutoff values is a cost-effective strategy. Received: May 2, 2014; Revised: July 18, 2014; Accepted: August 28, 2014 The Author Published by Oxford University Press. All rights reserved. For Permissions, please journals.permissions@oup.com. 1 of 10 Downloaded from

2 2 of 10 JNCI J Natl Cancer Inst, 2015, Vol. 107, No. 1 Based on consistent evidence that women with ovarian cancer experience improved clinical outcomes when initial surgical care is provided by a gynecologic oncologist (1 9), both the American Congress of Obstetricians and Gynecologists (ACOG) and the National Comprehensive Cancer Network (NCCN) recommend primary surgery for an ovarian malignancy by a subspecialist whenever possible (1,8,10). Because many adnexal masses are benign, there is a variety of strategies to help decide which women are most likely to benefit from initial surgery by a subspecialist. Two commercial biomarker-based algorithms, the Risk of Malignancy Algorithm (ROMA; Fujerebio Diagnostics, Malvern, PA) and the Multivariate Index Assay (MIA; Vermillion, Austin, TX), are currently on the market. Comparing the performance of the available referral algorithms using published studies is difficult because of differences in chosen diagnostic test cutoffs and study populations. In the absence of direct comparative effectiveness studies, computer simulation and decision modeling provide a formal way to estimate the likely costs and outcomes of different strategies for management of women with an adnexal mass. Methods Model We constructed a modified Markov decision analytic model using TreeAge Pro 2013 software (Williamstown, MA) to compare, from a societal perspective, the estimated costs and outcomes of five strategies to help clinicians decide which women with an adnexal mass requiring surgery would most benefit from subspecialist referral: 1) ACOG referral guidelines (ACOG) are stratified by menopausal status and based on clinical findings (ascites, evidence of abdominal or distant metastases, a nodular or fixed mass if postmenopausal) and serum levels of CA125 (1,11,12). 2) The Multivariate Index Assay (MIA; Vermillion, Austin, TX) is a proprietary algorithm incorporating serum levels of multiple biomarkers, including CA125-II, transferrin, transthyretin, apolipoprotein A1, and beta 2 microglobulin, and menopausal status, to generate an ovarian malignancy risk score (low probability or high probability) (13 15). 3) The Risk of Malignancy Algorithm (ROMA; Fujerebio Diagnostics, Malvern, PA) utilizes levels of CA125 and HE4 and the patient s menopausal status to classify patients as high- or low-risk for malignancy (16 20). 4) CA125 alone as a determinant of referral, with diagnostic threshold changed to prioritize test sensitivity, was modeled using the data from a previously published validation study of 389 women who underwent surgery for pelvic masses (21). 5) Referral of all women to a subspecialist without testing (Refer All). Referral rates appear to be high even when algorithms indicate a low risk of malignancy (22), suggesting that inclusion of a strategy in which additional tests are not performed is reasonable. The decision tree is depicted in Supplementary Figure 1 (available online). Because test characteristics and cancer prevalence vary markedly by menopausal status, we created separate models for premenopausal and postmenopausal women. The models were run as two-dimensional Monte Carlo microsimulations (23) using cohorts of women (the approximate number of women in each menopausal category who undergo surgery for an adnexal mass in the United States annually, estimated as described below). At the start of each simulation, an individual patient s age and physical distance from a generalist obstetrician-gynecologist and the closest subspecialist were drawn from the appropriate distribution. The model was then run times for each individual, drawing from distributions that incorporate uncertainty in the estimates of the other parameters in the model (Table 1). We estimated mean costs in 2013 US dollars and life expectancy, along with 95% confidence intevals (CIs). Assumptions included: 1) Test or algorithm results of highrisk or suspicious for malignancy lead to referral to a subspecialist for oophorectomy, while a negative/low-risk result leads to oophorectomy by the primary gynecologist. 2) Salpingooophorectomy not requiring surgical staging is performed using minimally invasive techniques in 50% of cases. 3) If ovarian cancer is diagnosed at surgery, one-third of primary surgical procedures and one-third of restaging procedures are performed using a minimally invasive approach; an approach considered acceptable in select, clinically early stage cases (8). 4) All falsenegative tests (initial surgery performed by a generalist) result in postoperative subspecialist referral, with a CT scan performed, followed by restaging/debulking surgery (immediately or following initiation of chemotherapy) in 50% of cases. 5) The average postsurgical treatment, including chemotherapy, is similar for women with ovarian cancer no matter who performed the initial surgery. Because the costs and impact on quality of life of this postsurgical cancer treatment are not expected to be different on average, these were not included in the analysis. 6) Recurrence rates after 80 months are independent of specialty of the original surgeon. Clinical Outcomes Probability estimates were derived from the literature and characterized as beta distributions (Table 1). The test characteristics of the modified ACOG guidelines, ROMA, and MIA were modeled using published results of three prospective, multi-institutional trials, each including over 450 women who presented with an ovarian mass and underwent surgery (15,20,22). To account for the inverse correlation between sensitivity and specificity, sensitivity was also characterized as a function of specificity and the diagnostic odds ratio, as described by Genders et al. (24). Because the model is specifically concerned with the outcomes of ovarian cancer and not metastatic cancer from other primary sites, test characteristic estimates were derived from published cohorts of benign masses and primary ovarian malignancies. Because the prevalence of ovarian malignancy was fairly similar across studies within pre- and postmenopausal categories, prevalence by menopausal status was estimated by pooling data from validation studies (Table 1) (15,20,22). The performance of CA125 alone was modeled using data from a previously published validation study of 389 women with pelvic masses who underwent surgery (21). Malignancies metastatic to the ovary were excluded, leaving 173 premenopausal and 184 postmenopausal cases. Borderline tumors were considered benign for calculation of test characteristics, as these lesions seldom result in clinically significant mortality. We used CA125 cutoffs for referral that approximated the reported sensitivity of the MIA assay (postmenopausal 15 U/mL, premenopausal 22 U/mL) (14,15). The age distribution of premenopausal and postmenopausal women were derived from incident cases in SEER or the National Database of State Cancer Registries from , using age 50 as the threshold for postmenopausal (25). Disease-specific Downloaded from

3 L. J. Havrilesky et al. 3 of 10 Table 1. Clinical estimates Parameter Numerator/ denominator for calculation of value Value (95% confidence intervals) Distribution type N Prevalence ovarian malignancy (%) Source Postmenopausal women Probability of ovarian cancer given pelvic mass 169/ (0.23 to 0.30) beta (15,20,22) MIA (14,15) Sensitivity 39/ (0.85 to 1.00) beta Specificity 72/175* 0.41 (0.33 to 0.49) beta Diagnostic odds ratio 7.90 (4.41 to 11.40) lognormal ACOG (22) Sensitivity 71/ (0.88 to 0.99) beta Specificity 93/ (0.49 to 0.64) beta Diagnostic odds ratio (13.63 to 19.90) lognormal ROMA (19 21) Sensitivity 46/ (0.82 to 0.98) beta Specificity 114/ (0.69 to 0.83) beta Diagnostic odds ratio (23.32 to 29.72) lognormal CA (21) Sensitivity 79/ (0.93 to 1.00) beta Specificity 62/ (0.51 to 0.69) beta Diagnostic odds ratio (12.75 to ) lognormal Premenopausal women Probability of ovarian cancer given pelvic mass 66/ (0.07 to 0.11) beta (15,20,22) MIA (14,15) Sensitivity 22/ (0.87 to 1.00) beta Specificity 156/254* 0.61 (0.55 to 0.67) beta Diagnostic odds ratio (19.24 to 28.43) lognormal ACOG modified (22) Sensitivity 18/ (0.49 to 0.84) beta Specificity 147/ (0.71 to 0.83) beta Diagnostic odds ratio 6.60 (3.58 to 9.63) lognormal ROMA (19 21) Sensitivity 13/ (0.62 to 1.00) beta Specificity 173/ (0.65 to 0.76) beta Diagnostic odds ratio (7.42 to 14.70) lognormal CA (21) Sensitivity 17/ (0.80 to 1.00) beta Specificity 94/ (0.53 to 0.68) beta Diagnostic odds ratio (3.79 to 97.5) lognormal Hazard ratio for disease specific survival No Gynecologic Oncologist N/A 1 (referent) Gynecologic Oncologist N/A 0.9 (0.78 to 1.03) lognormal (27) Mean distance from patient zipcode to site of surgery by a generalist Mean distance from patient zipcode to closest subspecialist practice (postmenopausal estimate) Mean distance from patient zipcode to closest subspecialist practice (premenopausal estimate) N/A 9.8 (0.29 to 55.1) lognormal 11,854 State inpatient databases (see Supplemental Digital Content 2) N/A 21.9 (2.7 to 82.8) lognormal 278 SEER-Medicare (see Supplemental Digital Content 2) N/A 28.2 (2.2 to 123.6) lognormal 11,854 State inpatient databases (see Supplemental Digital Content 2) * Statistics calculated for all pelvic malignancies detected, as ovarian malignancy alone was not available for this cohort. ACOG = American Congress of Obstetricians and Gynecologists; MIA = Multivariate Index Assay; N/A = not applicable; ROMA = Risk of Malignancy Algorithm; SEER = Surveillance, Epidemiology, and End Results Program. Distances in miles (see Methods). survival for women treated without subspecialist care was derived from a study of 1491 women treated for stage IC to stage IV ovarian cancer (26). The hazard ratio characterizing the effect of subspecialist referral on survival was derived from a Cochrane meta-analysis of three studies and over 9000 patients; five-year survival was statistically significantly higher (0.90, Downloaded from

4 4 of 10 JNCI J Natl Cancer Inst, 2015, Vol. 107, No. 1 95% CI = 0.82 to 0.99) when treatment was received at hospitals where gynecologic oncologists were on staff (9). The adjusted hazard ratio, modeled to represent the effect of subspecialist surgeon on disease-specific survival (DSS), was consistent with prior estimates of the survival benefit attributable to initial surgery by a subspecialist (4 6,27). We estimated undiscounted and discounted (using a 3% discount rate) life expectancy after 80 months for each age group using US life tables for women (28). measure of effectiveness and compared the effect of different values of willingness to pay (WTP; expressed as Total referrals per additional five-year survivor ). Second, we generated estimates of the annual number of cases performed by generalists and oncologists using each strategy (Supplementary Methods 2, available online). Assuming ACOG guidelines are the current referral standard, we estimated the average annual increase or decrease in cases per surgeon under the four alternatives. This study was declared exempt by the Duke University Institutional Review Board. Costs of Care Costs of surgical procedures, inpatient and outpatient postoperative care, and diagnostic tests were obtained using national Medicare reimbursements. The cost of hospice care for ovarian cancer, derived from Medicare data, was assigned to deaths from ovarian cancer (29) (Table 2) and standardized to 2013 values using the medical component of the Consumer Price Index ( accessed Novermber 7, 2013). Costs of Referral The target population for these algorithms is women for whom surgical intervention for an adnexal mass is planned; we assumed that imaging studies would not be repeated during a preoperative subspecialist visit. Referral prior to oophorectomy resulted in the costs of preoperative laboratory testing at the subspecialist s surgical facility, consultation with a subspecialist, an out of pocket copay of 20% of the total Medicare reimbursement value, round-trip travel, food, and parking (Table 2). We assumed that women referred to a subspecialist after oophorectomy by a generalist for unsuspected ovarian cancer would also undergo a CT scan prior to a decision regarding a second surgical procedure; 50% of these women would undergo additional staging/debulking surgery. Estimation of Travel Distance We used several data sources to estimate costs of travel to a generalist or subspecialist (see Supplementary Methods 1, available online). We estimated the distance between the addresses of women with adnexal masses who underwent surgery and the closest generalist and gynecologic oncology practices using the Surveillance, Epidemiology and End Results (SEER) Medicare linked database, the State Inpatient Databases (SID) and practice zip codes provided by the Society of Gynecologic Oncology (SGO). Measure of Cost-effectiveness The primary measure of cost-effectiveness was cost per year of life-saved (YLS); costs and life expectancy were discounted at 3% annual rate. Sensitivity Analysis Uncertainty was captured by performing a probabilistic analysis. Because Refer All might not be a practical option in many cases, we also conducted analyses comparing only CA125, ACOG, ROMA, and MIA. We explored the impact of each strategy on oncologist workload in two ways. First, we ran the model using total referrals to subspecialist as the measure of cost and ovarian cancer deaths prevented (based on 80-month survival) as the Results Cost-effectiveness in Postmenopausal Women Mean costs were lowest for CA125 and highest for MIA, with considerable overlap between confidence intervals (Table 3). Ovarian cancer mortality was lower, and life expectancy was slightly higher for Refer All than for biomarker strategies. CA125 had slightly higher life expectancy than MIA, ACOG, and ROMA, again with considerable overlap. Based on mean values alone, the incremental cost-effectiveness ratio for Refer All was $9423/ YLS compared with CA125. ACOG and ROMA were dominated (more expensive and less effective) by CA125, while MIA was dominated by Refer All. Figure 1, a cost-effectiveness acceptability curve, illustrates the effect of the uncertainty in the parameter estimates on costeffectiveness. The y-axis depicts the probability that a strategy is optimal (most cost-effective) at a given willingness-to-pay threshold, shown on the x-axis. Refer All is the optimal strategy in up to 95% of simulations as the WTP approaches $ / YLS. CA125 is the optimal strategy in over 50% of simulations at WTP under $15 000/YLS but remains the optimal strategy in less than 10% of simulations at WTP over $50 000/YLS. The other biomarker-based algorithms are each optimal in less than 10% of simulations over the range of WTP thresholds. Cost-effectiveness acceptability curves, postmenopausal women. Willingness to pay threshold (WTP) threshold expressed in $/year of life saved (YLS). Costs are in 2013 US dollars. ACOG = American Congress of Obstetricians and Gynecologists; MIA = Multivariate Index Assay; ROMA = Risk of Malignancy Algorithm. When only the four test algorithms were compared, CA125 dominated ACOG, ROMA, and MIA. In acceptability curve analysis, there is an 80% probability that CA125 is the optimal strategy across the entire WTP range. At a WTP of $ /YLS, the probability that each of the other strategies is optimal is: ROMA-12%; ACOG-5%; MIA-2%. Cost-effectiveness in Premenopausal Women There was substantial overlap in the confidence intervals for both costs and effectiveness between strategies. Mean costs were lowest for CA125, with Refer All more effective and more expensive, at a cost of $10 644/YLS. ACOG and ROMA were dominated by CA125, while MIA was dominated by Refer All (Table 3). In acceptability curve analysis, Refer All is preferred in over 50% of simulations at WTP thresholds above approximately $20 000/ YLS and in 90% of simulations at a threshold of $ /YLS (Figure 2). When excluding Refer All, the mean incremental cost-effectiveness ratio of MIA compared to CA125 was $ /YLS. ACOG and ROMA were dominated by CA125. As the willingness-to-pay Downloaded from

5 L. J. Havrilesky et al. 5 of 10 Table 2. Direct and indirect costs of tests, procedures, and medical care* Item Code (CPT/DRG) Cost (2013 US dollars) Source Biomarker tests CA January 2013 clinical diagnostic laboratory fee schedule (download available at HE January 2013 clinical diagnostic laboratory fee schedule (download available at MIA (accessed March 15, 2013) ROMA Estimate Costs of surgical procedures Minimally invasive BSO Open BSO Open staging for ovarian cancer Open debulking for ovarian cancer Minimally invasive pelvic and aortic lymph node dissection Outpatient surgical facility reimbursement Inpatient hospital reimbursement, ovarian cancer surgery Inpatient hospital reimbursement, benign BSO (accessed October 15, 2011 and inflated to 2013 dollars) (accessed October 15, 2011 and inflated to 2013 dollars) (accessed October 15, 2011 and inflated to 2013 dollars) Costs of initial referral to subspecialist Subspecialist consultation visit Copay for subspecialist visit N/A 40 org/2006/03/ccas html (accessed April 13, 2013) Travel to subspecialist N/A 5 See Methods Food, parking N/A 25 Estimate Complete blood count January 2013 clinical diagnostic laboratory fee schedule (download available at Basic metabolic panel January 2013 clinical diagnostic laboratory fee schedule (download available at Costs of referral following surgery by a generalist CT of chest, abdomen, pelvis Cost of end-of-life care with hospice Mean, $ (SD) (48 067) (29) * ACOG = American Congress of Obstetricians and Gynecologists; BSO = bilateral salpingo-oophorectomy; CPT = Common Procedural Terminology; DRG = Diagnosis- Related Group; MIA = Multivariate Index Assay; N/A = not applicable; ROMA = Risk of Malignancy Algorithm. Downloaded from

6 6 of 10 JNCI J Natl Cancer Inst, 2015, Vol. 107, No. 1 Table 3. Ovarian cancer deaths, costs, life expectancy, and cost-effectiveness of four strategies for managing an adnexal mass Strategy Ovarian cancer deaths per 1000 women Mean cost* (95% CI), $ Mean effectiveness* (95% CI), y Incremental cost-effectiveness ratio, $/life-years Postmenopausal CA (148 to 220) ( to ) (16.23 to 17.60) -- ACOG (149 to 220) ( to ) (16.23 to 17.59) Dominated ROMA (150 to 220) ( to ) (16.21 to 17.58) Dominated REFER ALL (147 to 219) ( to ) (16.24 to 17.61) 9423 MIA (148 to 220) ( to ) (16.23 to 17.58) Dominated Premenopausal CA (42 to 83) 9876 (8699 to ) (27.97 to 29.15) -- ACOG 62.3 (42 to 85) 9892 (8699 to ) (27.93 to 29.13) Dominated ROMA 61.9 (42 to 84) 9897 (8691 to ) (27.95 to 29.13) Dominated REFER ALL 61.0 (42 to 83) 9999 (8819 to 1 309) (27.97 to 29.16) MIA 61.3 (42 to 83) (9156 to ) (27.97 to 29.16) Dominated * Listed in 2013 US dollars and discounted at 3% annually. ACOG = American Congress of Obstetricians and Gynecologists; CI = confidence interval; MIA = Multivariate Index Assay; ROMA = Risk of Malignancy Algorithm. Figure 1. Cost-effectiveness acceptability curves, postmenopausal women. Willingness to pay threshold (WTP) threshold expressed in $/year of life saved (YLS). Costs are in 2013 US dollars. ACOG = American Congress of Obstetricians and Gynecologists; MIA = Multivariate Index Assay; ROMA = Risk of Malignancy Algorithm. threshold increases, the probability that CA125 is the most costeffective option is 65% to 80%; MIA is favored in approximately 30% of the simulations at a threshold of $ /YLS. Physician Workload Figure 3 depicts the effects of uncertainty about test characteristics and survival advantage on the optimal strategy expressed as total cases performed by a subspecialist per ovarian cancer death prevented for postmenopausal (Figure 3A) and premenopausal (Figure 3B) women. As willingness to pay increases, and particularly above 120 referrals per cancer death prevented, the more sensitive strategies become optimal. The convergence of the strategies as willingness to pay increases reflect the high degree of overlap in sensitivity and specificity estimates between the tests. Downloaded from

7 L. J. Havrilesky et al. 7 of 10 Table 4 depicts the estimated number of surgeries performed by generalists and oncologists under each strategy, based on the mean sensitivity and specificity estimates in Table 1, as well as the estimated change in annual number of cases per surgeon with each strategy assuming ACOG as the reference. ROMA results in six fewer cases per subspecialist annually compared with ACOG. CA125, MIA, and Refer All result in 9, 18, and 73 additional cases per subspecialist annually, respectively. Discussion In May 2013, the Society of Gynecologic Oncology (SGO) released a statement on its website regarding the MIA: the test may be useful in identifying women who should be referred to a subspecialist. Recent data have suggested that the OVA1 test along with physician clinical assessment may improve detection rates of malignancies among women with pelvic masses planning surgery (30). This statement and the clinical studies that preceded Food and Drug Administration approval of the MIA were heavily publicized, and the MIA test heavily marketed to gynecologists (31). The primary rationale for a test-based strategy to decide which women with an adnexal mass need subspecialist referral is to increase the likelihood that women with a true ovarian malignancy will undergo appropriate treatment with improved outcomes (1,3-10) (a function of the strategy s sensitivity), while minimizing the number of unnecessary referrals (a function of specificity). The determinants of the optimal strategy must necessarily include efficiency, particularly in an era of increased awareness of healthcare costs. Model-based analyses are helpful in understanding the effect of the inherent tradeoff between sensitivity and specificity on clinical outcomes and costs and in illustrating how the optimal choice may vary depending on the value patients, clinicians, and society place on these outcomes. Our model suggests that there is substantial overlap in the estimated costs and outcomes resulting from the use of any particular algorithm. In the absence of any systemic effects on access or waiting times, referral of all women with an adnexal mass requiring surgery to a subspecialist would be well within accepted thresholds of cost-effectiveness and would maximize expected survival. Kim et al. previously reported in a societal perspective cost-minimization analysis that referral of all patients to a subspecialist was less costly than the use of the MIA algorithm (32). While the prior model differed in some assumptions and methodological aspects from ours, its conclusions are similar. The validity of our findings is dependent on the accuracy of assumptions regarding the process of care with and without referral. For example, we did not account for additional costs that might be incurred for family members, such as time lost from work or hotel stays that are associated with longer distances, and it is possible that universal referral is not as Figure 2. Cost-effectiveness acceptability curves, premenopausal women. Willingness to pat threshold (WTP) expressed in $/year of life saved (YLS). Costs are in 2013 US dollars. ACOG = American Congress of Obstetricians and Gynecologists; MIA = Multivariate Index Assay; ROMA = Risk of Malignancy Algorithm. Downloaded from

8 8 of 10 JNCI J Natl Cancer Inst, 2015, Vol. 107, No. 1 Figure 3. Cost-effectiveness acceptability curves, expressed in total cases referred to subspecialists per additional ovarian cancer death prevented. A) Postmenopausal women, B) Premenopausal women. ACOG = American Congress of Obstetricians and Gynecologists; MIA = Multivariate Index Assay; ROMA = Risk of Malignancy Algorithm. Table 4. Estimated number of salpingo-oophorectomies and change relative to ACOG guidelines in annual cases performed by subspecialists (n = 981) and generalist gynecologists (n = ) using each strategy* Category ACOG CA125 ROMA MIA Refer All Benign Gyn Gyn Onc Change (cases/surgeon) Gyn Ref Gyn Onc Ref Cancer Gyn Gyn Onc Change (cases/surgeon) Gyn Ref Gyn Onc Ref Total Gyn Gyn Onc Change (cases/surgeon) Gyn Ref Gyn Onc Ref * Key assumptions: 1) annual cases of ovarian cancer ( 2) 75% of all ovarian cancers are postmenopausal ( 3) Prevalence of ovarian cancer among all adnexal masses requiring surgery is 26% in postmenopausal and 9% among premenopausal population (pooled prevalence from studies used to construct decision model). ACOG = American Congress of Obstetricians and Gynecologists; MIA = Multivariate Index Assay; Ref = referent; ROMA = Risk of Malignancy Algorithm. cost-effective for this subset of patients. Patients may also prefer to have their surgery performed by a gynecologist with whom they have a longstanding relationship and at a facility closer to home. Other patients may have different preferences for an acceptable degree of risk of cancer, and the resulting anxiety may affect quality of life until a definitive diagnosis is made. Methods such as conjoint analysis are very useful for identifying patient preferences for these nonoutcome attributes of care (33), including preferences for local care (34), and such studies would be extremely helpful to inform future analyses of the relative value of referral strategies. The finding that universal referral is likely to be cost-effective in both pre- and postmenopausal women is dependent on adequate capacity to handle additional cases among subspecialists and the facilities in which they operate. Using ACOG guidelines as the baseline, universal referral would result in an increase of 73 cases annually per oncologist; impact on waiting times for appointments and surgery is unclear. If adding additional capacity in terms of more oncologists, support staff, and facilities were necessary to implement universal referral, these initial investment costs would need to be included in the analysis. If universal referral is not a practical option, there is considerable uncertainty about which of the test-based algorithms is most efficient, particularly for premenopausal women. For postmenopausal women, there is an 80% probability that CA125 is the most efficient strategy, while the chance that MIA is costeffective compared with the remaining algorithms is 2% or less. For premenopausal women, there is a higher degree of uncertainty, but with CA125 still the optimal strategy in up to 80% of simulations. Downloaded from

9 L. J. Havrilesky et al. 9 of 10 The sensitivity and specificity of each test or referral algorithm is determined by the choice of test cutoff for specific biomarkers, such as CA125. If maximizing sensitivity is the goal, then simply choosing a different cutoff value for CA125 appears to result in similar performance to more expensive proprietary tests. In modeling CA125 alone, we adjusted cutoff values downward (15 U/mL for postmenopausal and 22 U/mL for premenopausal cohorts), with resulting sensitivities above 94% and specificities at 60% in both cohorts. These test characteristics are similar to those of MIA, a test costing over ten times as much. The MIA s algorithm includes CA125, at an unspecified threshold; because the details of the algorithm are proprietary, the relative contribution of the various components of the algorithm to the overall performance is unclear. Like any model-based analysis, ours is limited by both the available data and the assumptions made. Most importantly, the modeled effectiveness data is based on assumption of a survival advantage for women who undergo surgery by a subspecialist. We derived this estimate from a Cochrane meta-analysis (9); individual studies have found even greater survival benefit (4), which would make Refer All even more efficient. Although referring all patients to a gynecologic oncologist may increase anxiety in some patients, all other potential effects of different testing strategies anxiety about the potential for a false-negative test result, decisional regret in the event of a false-negative test result, and the effect of a second surgical procedure, would all favor Refer All if quality-adjusted life expectancy were the primary measure of effectiveness. Based on the currently available evidence, our analysis suggests that universal referral is a cost-effective strategy for management of women with an adnexal mass if there is adequate capacity. The value of test-based algorithms lies mostly in their impact on surgeon volume and, potentially, access to care by patients. Funding This study was funded by the Charles Hammond Research Fund at Duke University. Notes The authors acknowledge the Society of Gynecologic Oncology for making anonymous practice zip code data for full members of the society available. The funding source had no role in the design and conduct of the study, collection, management, analysis, or interpretation of the data, preparation, review, or approval of the manuscript, nor the decision to submit the manuscript for publication. LH, MD, GS, JB, LC, and EM all declare no conflict of interest or financial interest with regard to this work. Author contributions: LH and EM participated in conception and design, data acquisition, analysis, and interpretation, drafting and revising the, and final approval. MD and LC participated in data acquisition, analysis, and interpretation, drafting and revising the, and final approval. GS and JB participated in conception and design, drafting and revising the, and final approval. References 1. ACOG Committee Opinion: number 280, December The role of the generalist obstetrician-gynecologist in the early detection of ovarian cancer. Obstet Gynecol. 2002;100(6): NIH consensus conference. Ovarian cancer. Screening, treatment, and follow-up. NIH Consensus Development Panel on Ovarian Cancer. JAMA. 1995;273(6): Myers ER, Bastian LA, Havrilesky LJ, et al. Management of adnexal mass. Evid Rep Technol Assess (Full Rep). 2006;(130): Available at: entrez/query.fcgi?cmd=retrieve&db=pubmed&dopt=citation &list_uids= Engelen MJ, Kos HE, Willemse PH, et al. Surgery by consultant gynecologic oncologists improves survival in patients with ovarian carcinoma. Cancer. 2006;106(3): Giede KC, Kieser K, Dodge J, et al. Who should operate on patients with ovarian cancer? An evidence-based review. Gynecol Oncol. 2005;99(2): Carney ME, Lancaster JM, Ford C, et al. A population-based study of patterns of care for ovarian cancer: who is seen by a gynecologic oncologist and who is not? Gynecol Oncol. 2002;84(1): Im SS, Gordon AN, Buttin BM, et al. Validation of referral guidelines for women with pelvic masses. Obstet Gynecol. 2005;105(1): Morgan RJ Jr, Alvarez RD, Armstrong DK, et al. NCCN Clinical Practice Guidelines in Oncology: epithelial ovarian cancer. J Natl Compr Canc Netw. 2011;9(1): Woo YL, Kyrgiou M, Bryant A, et al. Centralisation of services for gynaecological cancers - a Cochrane systematic review. Gynecol Oncol. 2012;126(2): NIH consensus conference. Ovarian cancer. Screening, treatment, and follow-up. NIH Consensus Development Panel on Ovarian Cancer. JAMA. 1995;273(6): Dearking AC, Aletti GD, McGree ME, et al. How relevant are ACOG and SGO guidelines for referral of adnexal mass? Obstet Gynecol. 2007;110(4): ACOG Committee Opinion: number 477, March The role of the generalist obstetrician-gynecologist in the early detection of ovarian cancer. Obstet Gynecol. 2011;117(3): Zhang Z, Bast RC Jr, Yu Y, et al. Three biomarkers identified from serum proteomic analysis for the detection of early stage ovarian cancer. Cancer Res. 2004;64(16): Ueland FR, Desimone CP, Seamon LG, et al. Effectiveness of a multivariate index assay in the preoperative assessment of ovarian tumors. Obstet Gynecol. 2011;117(6): Bristow RE, Smith A, Zhang Z, et al. Ovarian malignancy risk stratification of the adnexal mass using a multivariate index assay. Gynecol Oncol. 2013;128(2): Bast RC Jr, Klug TL, St John E, et al. A radioimmunoassay using a monoclonal antibody to monitor the course of epithelial ovarian cancer. N Engl J Med. 1983;309(15): Jacobs I, Bast RC Jr. The CA 125 tumour-associated antigen: a review of the literature. Hum Reprod. 1989;4(1): Moore RG, Brown AK, Miller MC, et al. The use of multiple novel tumor biomarkers for the detection of ovarian carcinoma in patients with a pelvic mass. Gynecol Oncol. 2008;108(2): Moore RG, McMeekin DS, Brown AK, et al. A novel multiple marker bioassay utilizing HE4 and CA125 for the prediction of ovarian cancer in patients with a pelvic mass. Gynecol Oncol. 2009;112(1): Moore RG, Miller MC, Disilvestro P, et al. Evaluation of the diagnostic accuracy of the risk of ovarian malignancy algorithm in women with a pelvic mass. Obstet Gynecol. 2011;118(2 Pt 1): Van Gorp T, Cadron I, Despierre E, et al. HE4 and CA125 as a diagnostic test in ovarian cancer: prospective validation of the Risk of Ovarian Malignancy Algorithm. Br J Cancer. 2011;104(5): Downloaded from

10 10 of 10 JNCI J Natl Cancer Inst, 2015, Vol. 107, No Ware Miller R, Smith A, DeSimone CP, et al. Performance of the American College of Obstetricians and Gynecologists ovarian tumor referral guidelines with a multivariate index assay. Obstet Gynecol. 2011;117(6): Briggs AH, Weinstein MC, Fenwick EA, et al. Model parameter estimation and uncertainty analysis: a report of the ISPOR- SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32(5): Genders TS, Meijboom WB, Meijs MF, et al. CT coronary angiography in patients suspected of having coronary artery disease: decision making from various perspectives in the face of uncertainty. Radiology. 2009;253(3): United States Cancer Statistics: Incidence, WONDER Online Database. United States Department of Health and Human Services, Centers for Disease Control and Prevention and National Cancer Institute. Available at: Accessed July 25, Mutch DG, Prat J FIGO staging for ovarian, fallopian tube and peritoneal cancer. Gynecol Oncol. 2014;133(3): Chan JK, Kapp DS, Shin JY, et al. Influence of the gynecologic oncologist on the survival of ovarian cancer patients. Obstet Gynecol. 2007;109(6): Arias E. United States life tables, In: National vital statistics reports: National Center for Health Statistics Pyenson B, Connor S, Fitch K, et al. Medicare cost in matched hospice and non-hospice cohorts. J Pain Symptom Manage. 2004;28(3): Society of Gynecologic Oncology. Multiplex Serum Testing for Women with a Pelvic Mass. position-statements-2/multiplex-serum-testing-for-womenwith-pelvic-mass/. Accessed August 21, Vermillion. Vermillion s OVA1 receives new statement by Society of Gynecologic Oncology. Available at: phoenix.zhtml?c=121814&p=irol-newsarticle&id= Accessed January 15, Kim KH, Zsebik GN, Straughn JM Jr, et al. Management of complex pelvic masses using a multivariate index assay: a decision analysis. Gynecol Oncol. 2012;126(3): Brazier J. Measuring and valuing health benefits for economic evaluation. Oxford; New York, NY: Oxford University Press; Ryan M, Farrar S. Using conjoint analysis to elicit preferences for health care. BMJ. 2000;320(7248): Downloaded from