Predictive Performance Evaluation

Transcription

1 Predictive Performance Evaluation Clinical Performance of the 4Kscore Test to Predict High-grade Prostate Cancer at Biopsy: A Meta-analysis of US and European Clinical Validation Study Results Stephen M. Zappala, MD, 1 Peter T. Scardino, MD, 2,3 David Okrongly, PhD, 4 Vincent Linder, PhD, 5 Yan Dong, PhD 5 1Tufts University School of Medicine, Boston, MA; 2 Memorial Sloan Kettering Cancer Center, New York, NY; 3 Weill Cornell Medical College, New York, NY; 4 OPKO Diagnostics, Miami, FL; 5 OPKO Diagnostics, Woburn, MA The 4Kscore Test (OPKO Diagnostics, Woburn, MA) is a blood test utilized prior to a prostate biopsy to determine a patient s risk of high-grade prostate cancer (PCa) should the biopsy be performed, thus providing critical information in the clinical management of men with a suspicious prostate-specific antigen value or digital rectal examination result. Multiple US and European clinical studies confirmed that a prebiopsy 4Kscore Test has a high degree of discrimination for a subsequent discovery of high-grade (Gleason score 7) PCa. The aim of this study was to evaluate the predictive accuracy of the 4Kscore Test to discriminate between patients with and without high-grade PCa based on published clinical validation studies. A systematic review and meta-analysis of the eligible 4Kscore Test clinical validation studies was conducted. The pooled area under the curve (AUC) of the 4Kscore Test as reported from all the studies, and the heterogeneity among these studies were analyzed and repeated for subgroups of the studies. Twelve clinical validation studies were included in the meta-analysis, comprising a total of 11,134 patients. The pooled AUC to discriminate for high-grade PCa for all 12 studies was 0.81 (fixed effects 95% CI, ). Restricting the analysis to the six publications that used the contemporary 4Kscore Test algorithm led to very similar results (AUC 0.81; 95% CI, ). Heterogeneity was high among all of the 12 studies, as well as among the six publications that used the contemporary 4Kscore Test (Cochrane s Q test, p for both); however, in both cases, after exclusion of a single outlying study with a much lower AUC, heterogeneity was no longer significant (p and p ). The pooled estimate of 4Kscore Test discrimination (AUC) for high-grade PCa is.0.80, and is consistent across multiple US and European clinical validation studies. [Rev Urol. 2017;19(3): doi: /riu0776] 2017 MedReviews, LLC Vol. 19 No Reviews in Urology 149

2 Clinical Performance of the 4Kscore Test to Predict High-grade PCa at Biopsy continued KEY WORDS 4Kscore Test Biomarkers High-grade prostate cancer Early detection Biopsy The 4Kscore Test (OPKO Diagnostics, Woburn, MA), also known in the scientific literature as the four-kallikrein panel, is a commercially available, prebiopsy blood test to predict a patient s risk of high-grade prostate cancer (PCa) should the biopsy be performed. The 4Kscore Test incorporates four kallikrein protein measurements and clinical information consisting of age, DRE result, and history of prior negative biopsy result. The four kallikrein biomarkers include three different isoforms of PSA (total, free, and intact PSA), and human kallikrein-related peptidase 2 (hk2). The laboratory information from the four kallikrein measurements is combined with the clinical information by the 4Kscore Test algorithm and provides the patient s individual risk or probability for high-grade PCa. The information provided by the 4Kscore Test is used by men with either a suspicious PSA level or DRE result for shared decision making with their physicians regarding whether to proceed with a prostate biopsy. The 4Kscore Test algorithm was originally developed and validated in retrospective studies of serum samples collected from cohorts that were part of the European Research Study on Prostate Cancer (ERSPC). 1 This algorithm (referred to in this study as the ERSPC algorithm) was later revised to the contemporary 4Kscore Test algorithm to account for fundamental changes in the standard of care occurring after the ERSPC samples were collected. These changes impacted both the detection rate and grading of PCa (the transition from (,7.5%). Stattin and associates 9 found that men with an elevated PSA level and a low 4Kscore Test (,7.5%) men with an elevated PSA level and a low 4Kscore Test (,7.5%) have a very low 20-year risk of developing metastatic PCa, and thus could safely avoid a prostate biopsy. sextant to 10-core or higher prostate biopsy, and changes in the pathologic definition of high-grade PCa). 2 These changes were validated in a prospective US validation study, 3 a 26-center study that demonstrated an area under the curve (AUC) of 0.82 for discriminating the presence of high-grade PCa with nearly perfect calibration of the risk predicted by the 4Kscore Test and the actual prostate biopsy results. Subsequent US 4, and European 5,6 validation studies also demonstrated similarly high AUC results. The 4Kscore Test has been included in the National Comprehensive the use of the 4Kscore Test in actual clinical practice resulted in a 65% biopsy rate reduction in a patient cohort referred for suspicion of PCa by conventional screening methods. 150 Vol. 19 No Reviews in Urology Cancer Network Prostate Cancer Early Detection guidelines since In a retrospective clinical utility study, Konety and colleagues 8 demonstrated that the use of the 4Kscore Test in actual clinical practice resulted in a 65% biopsy rate reduction in a patient cohort referred for suspicion of PCa by conventional screening methods. The study also showed that physicians and patients were more inclined to proceed with a prostate biopsy for those patients with the higher 4Kscore Test results ( 20%) and less inclined to biopsy men with low 4Kscore Test results have a very low 20-year risk of developing metastatic PCa, and thus could safely avoid a prostate biopsy. Voigt and colleagues 10 showed that reducing unnecessary prostate biopsies based on the 4Kscore Test could result in fewer negative biopsy results, fewer diagnoses of low-grade PCa, and reduced treatment or active surveillance of men with low-grade cancer, while still maintaining a high overall detection rate for high-grade PCa. These findings demonstrate that implementation of the 4Kscore Test can provide both improvement in patient care and substantial savings in healthcare costs. The 4Kscore Test has been the subject of numerous clinical studies in Europe and the United States. Herein we report a systematic review and meta-analysis to evaluate the performance of the 4Kscore Test in the prebiopsy setting across all eligible clinical validation studies and to evaluate the heterogeneity of the 4Kscore Test performance across these studies. Methods This meta-analysis was conducted following the guidelines of Preferred Reporting Items for Systematic Reviews and Meta- Analysis (PRISMA). 11 We carried out a systematic literature search on PubMed for all articles using

3 Clinical Performance of the 4Kscore Test to Predict High-grade PCa at Biopsy the search terms kallikrein panel and prostate cancer; 4Kscore Test; or four kallikrein and prostate cancer. The search results included all studies published through June 30, Studies that fulfilled all the following criteria were included in the meta-analysis: (1) case-control or cohort studies, (2) a prespecified 4Kscore Test algorithm used as the diagnostic test, and (3) the AUC for high-grade PCa upon biopsy was reported. One additional 366-patient clinical study, a prospective, multicenter, validation of the 4Kscore Test, was also identified from a recent academic conference presentation and was included in the meta-analysis. 4 The results of the literature search were also confirmed by one of the academic researchers involved with the test to ensure that the systematic review was inclusive of all eligible 4Kscore Test clinical studies (A. Vickers, personal communication). We then conducted a metaanalysis of the eligible publications using the AUC for high-grade PCa reported in each study. All studies reported AUCs from models either including DRE, without DRE, or both types of models (Table 1). The 4Kscore Test applied in the current clinical practice accepts DRE status of positive, negative, or not available. Therefore, in this meta-analysis, for those studies that reported AUCs from both models (with and without DRE included), we chose the AUC from the model that included DRE. Standard errors were calculated from the 95% CI reported for the AUCs in the eligible studies. We calculated the pooled AUC based on both the fixed-effects model and the random-effects model, and assessed the heterogeneity among the studies. Sensitivity analyses were performed by repeating the calculation of the pooled AUC and heterogeneity on (1) the prespecified subgroup of studies using the contemporary 4Kscore Test algorithm, (2) all studies and the subgroup of contemporary 4Kscore Test algorithm studies after excluding one study with obviously outlying AUC results, and (3) excluding the only study that is not currently published in a full article. Results The key word literature search identified 60 articles. After removing duplicates, 45 articles remained. Of these 45 articles, 27 were excluded because they are not clinical study articles. Seven clinical study articles did not pass the prespecified metaanalysis criteria and therefore were excluded (Figure 1), including the TABLE 1 Summary of Eligible 4Kscore Test a Clinical Studies Encompassing 11,134 Patients in the Validation Cohorts Study Country Study (N) Validation Cohort (N) Type 4Kscore Test Algorithm DRE Status in Algorithms Benchikh A et al 21 France Retrospective ERSPC Included Gupta A et al 22 Netherlands Retrospective ERSPC Included Vickers AJ et al 23 Sweden Retrospective ERSPC Both Vickers A et al 24 Netherlands Retrospective ERSPC Both Vickers AJ et al 25 Netherlands Retrospective ERSPC Both Kim EH et al 26 United States Retrospective ERSPC Both Nordström T et al 13 Sweden Retrospective Contemporary Not included Braun K et al 5 Sweden Retrospective Contemporary Both Bryant RJ et al 2 United Kingdom Retrospective Contemporary Not included Borque-Fernando A et al, Spain Prospective Contemporary Included Parekh DJ et al 3 United States Prospective Contemporary Included Punnen S et al 4 United States Prospective Contemporary Included Total 16,927 11,134 a The 4Kscore Test is manufactured by OPKO Diagnostics (Woburn, MA). DRE, digital rectal examination. Vol. 19 No Reviews in Urology 151

4 Clinical Performance of the 4Kscore Test to Predict High-grade PCa at Biopsy continued Figure 1. Literature search process. 60 records identified through PubMed searching; 1 record identified through additional source 46 records after duplicates removed 46 records screened 19 peer-reviewed publications assessed for eligibility 12 studies included in meta-analysis first 4Kscore Test clinical study published in This study reported on the development of the ERSPC 4Kscore Test algorithm, but was not a validation study. Eventually, 27 records excluded 20 review articles 1 commentary essay 6 not clinical studies 7 publications excluded 1 did not use 4Kscore as diagnostic test 4 studied outcomes different from high-grade cancer 1 has the same cohort as was used in another included article 1 was not a validation of a prespecified 4Kscore algorithm 11 eligible articles were retrieved from the literature search and 1 additional clinical study was identified from another source. 4 These 12 studies included a total of 16,927 patients from the United States and 5 different European countries; among them, 11,134 patients were involved in the validation of a prespecified 4Kscore Test algorithm (Table 1). Of the 12 studies, 5 studies used the 4Kscore Test models that included the DRE information, 2 studies used 4Kscore Test models that did not include the DRE information, and 5 studies included both types of models (Table 1). For the five studies with two models, the differences in AUC between the two models were very small, and we used the AUC of the model including DRE information in the meta-analysis. The pooled AUCs and heterogeneity analysis of all studies and subgroups are summarized in Table 2, and demonstrate that the pooled AUC is between 0.80 and 0.82 for all studies and subgroups. Specifically, the pooled AUC of the 4Kscore Test across all 12 clinical validation studies was 0.81 (fixed effects 95% CI, ; random effects 95% CI, ). The heterogeneity across studies was significant (p ). One study that utilized the TABLE 2 Summary of AUC and Heterogeneity Study Studies (N) Total N in Validation Studies AUC, Fixedeffects Estimate (95% CI) AUC, Randomeffects Estimate (95% CI) Heterogeneity Cochrane Q Test p Value All eligible 4Kscore Test a studies 12 11, ( ) 0.81 ( ) b Excluding Nordström T et al , ( ) 0.82 ( ) 0.08 Excluding Punnen S et al , ( ) 0.81 ( ),0.001 b All Contemporary 4Kscore Test studies ( ) 0.80 ( ) b Excluding Nordström T et al ( ) 0.82 ( ) 0.21 Excluding Nordström T et al 13 and Punnen S et al ( ) 0.82 ( ) 0.13 a The 4Kscore Test is manufactured by OPKO Diagnostics (Woburn, MA). b Indicates significant heterogeneity. AUC, area under the curve. 152 Vol. 19 No Reviews in Urology

5 Clinical Performance of the 4Kscore Test to Predict High-grade PCa at Biopsy STHLM2 cohort 13 appeared to be an outlier, with a much lower AUC (0.72; 95% CI, ) than the other 11 studies (AUC range, ). Exclusion of this study eliminated heterogeneity (p ) and led to an AUC of 0.82 (fixed effects 95% CI, ; random effects 95% CI, ). The ERSPC and contemporary 4Kscore Test studies were conducted over a 20-year timeframe on very different patient populations; however, after excluding a single study, 13 the remaining clinical studies showed highly consistent AUCs (Figure 2). Due to the differences between the ERSPC and contemporary 4Kscore Test algorithm, we conducted further prespecified subgroup analysis on the six studies (n ) that used the contemporary 4Kscore Test algorithm. The pooled AUC of these newer studies was not significantly changed compared with the pooled AUC for all 12 studies (fixed-effects AUC , 95% CI, ; random-effects AUC , 95% CI, ). However, the heterogeneity due A single published study of 531 men from the 26,712 STHLM2 cohort was found to be an outlier in our meta-analysis. This study concluded that there was no difference between the performance of the 4Kscore test and another test, the Prostate Health Index (phi). However, our statistical analyses raise concerns about this conclusion. to the study by Nordström and coworkers 13 stood out as an outlier (Cochrane Q test p ); again, excluding this study eliminated heterogeneity (p ) and yielded an AUC of 0.82 (95% CI, for both fixed-effects and randomeffects models). At the time of this publication, one study published by Punnen and coworkers 4 had only appeared as a conference abstract. When this study was removed, there was no impact on pooled AUC or the heterogeneity conclusions (Table 2). Discussion The 4Kscore Test has been validated in multiple European studies and the prospective US validation study (Table 1) and was the subject of two meta-analysis studies 14,15 and several reviews. 1,16-19 Our systematic meta-analysis is unique, as it encompassed all studies that were included in previous reviews or meta-analyses, in addition to new studies through Our metaanalysis is fundamentally different from that of Vickers and coworkers 14 in terms of the scope of patient population and evaluation method. Their study focused on patients with PSA in the 10- to 25-ng/mL range and patients with abnormal results on DRE, whereas our study covers all patients who are subject to prostate biopsy. The study by Vickers and coworkers 14 also used individual patient data for their meta-analysis, and our reported results are based on published data Figure 2. Forest plot of area under the curve (AUC) values in the 4Kscore Test clinical studies. The pooled AUCs were estimated based on the overall results from the 12 studies using a fixed-effects model and a random-effects model. The 4Kscore Test is manufactured by OPKO Diagnostics (Woburn, MA). Study AUC (95% Cl) Weight Benchikh et al., (0.81, 0.93) 3.95 Gupta et al., 2010 Vickers et al., 2010a Vickers et al., 2010b Vickers et al., 2010c Nordström et al., 2015 Braun et al., 2015 Bryant et al., 2015 Borque-Fernando et al., 2015 Parekh et al., 2015 Kim et al., 2017 Punnen et al., 2017 Overall (fixed effects estimate) Overall (random effects estimate) 0.87 (0.81, 0.94) (0.74, 0.86) (0.80, 0.87) (0.75, 0.85) (0.67, 0.77) (0.74, 0.83) (0.81, 0.88) (0.68, 0.91) (0.79, 0.85) (0.75, 0.82) (0.77, 0.86) (0.80, 0.83) (0.79, 0.83) Vol. 19 No Reviews in Urology 153

6 Clinical Performance of the 4Kscore Test to Predict High-grade PCa at Biopsy continued from individual studies, summarized using both fixed effects and random effects models. Our current meta-analysis reported here is also fundamentally different from Russo and associates, 15 whose conclusions on high-grade PCa were impacted by multiple deficiencies. 20 Our analysis included two studies published before 2016 that Russo and associates 15 failed to include, and the methodology used in this metaanalysis is substantially different from that used by Russo and associates. 15 Instead of calculating pooled sensitivity and specificity based on arbitrarily assigned cutoff points, we evaluated the in our meta-analysis. 14 This study concluded that there was no difference between the performance of the 4Kscore Test and another test, the Prostate Health Index (phi). However, our statistical analyses raise concerns about this conclusion. The results of our meta-analysis demonstrate reliable discrimination of the 4Kscore Test for high-grade PCa across multiple cohorts in the United States and in Europe (Table 1). The three prospective studies using the contemporary 4Kscore Test algorithm in routine laboratory service yielded highly consistent AUC. The most recent study, 4 which was performed in a cohort with a majority of African-American subjects, also Specifically, the pooled AUC of the 4Kscore Test across all 12 clinical validation studies was performance of the 4Kscore Test by calculating the pooled AUC based on the reported AUC and 95% CI of the AUC from each study. This strategy is preferred because the 4Kscore Test gives a continuous risk score from,1% to.95% that allows the physician and patient to act according to their own desired risk threshold. The predictive accuracy of the 4Kscore Test is therefore best evaluated by its AUC instead of the sensitivity and specificity at an arbitrary cutoff point. A single published study of 531 men from the 26,712 STHLM2 cohort was found to be an outlier resulted in an AUC consistent with the pooled AUC. Conclusions The pooled AUC of the 4Kscore Test for discrimination of highgrade PCa is above 0.80 for all the eligible studies and all subgroups in this meta-analysis. Despite the presence of one outlier study, the 4Kscore Test performance is highly consistent across the remaining 11 clinical validation studies involving over 10,000 subjects. This research was sponsored by OPKO Diagnostics, Woburn, MA. References 1. Punnen S, Pavan N, Parekh DJ. Finding the wolf in sheep s clothing: the 4Kscore Test is a novel blood test that can accurately identify the risk of aggressive prostate cancer. Rev Urol. 2015;17: Bryant RJ, Sjoberg DD, Vickers AJ, et al. Predicting high-grade cancer at ten-core prostate biopsy using four kallikrein markers measured in blood in the ProtecT study. J Natl Cancer Inst. 2015;107:djv Parekh DJ, Punnen S, Sjoberg DD, et al. A multi-institutional prospective trial in the USA confirms that the 4Kscore Test accurately identifies men with high-grade prostate cancer. Eur Urol. 2014; 68: Punnen S, Freedland S, Polascik T, et al. An independent, multi-institutional, prospective study in the Veterans Affairs Health System confirms the 4Kscore Test accurately predicts aggressive prostate cancer. J Urol. 2017;197(4 suppl):e1356-e1357. PD Braun K, Sjoberg DD, Vickers AJ, et al. A fourkallikrein panel predicts high-grade cancer on biopsy: independent validation in a community cohort. Eur Urol. 2016;69: Borque-Fernando A, Esteban-Escaño LM, Rubio- Briones J, et al. A preliminary study of the ability of the 4Kscore test, the Prostate Cancer Prevention Trial- Risk Calculator and the European Research Screening Prostate-Risk Calculator for predicting high-grade prostate cancer. Actas Urol Esp. 2016;40: Carroll PR, Parsons JK, Andriole G, et al. NCCN Guidelines Insights: Prostate Cancer Early Detection, Version 2. J Natl Compr Canc Netw. 2016;14: Konety B, Zappala SM, Parekh D, et al. The 4Kscore Test test reduces prostate biopsy rates in community and academic urology practices. Rev Urol. 2016;17: Stattin P, Vickers AJ, Sjoberg DD, et al. Improving the specificity of screening for lethal prostate cancer using prostate-specific antigen and a panel of kallikrein markers: a nested case control study. Eur Urol. 2015;68: Voigt JD, Dong Y, Linder V, Zappala S. Use of the 4Kscore test to predict the risk of aggressive prostate cancer prior to prostate biopsy: overall cost savings and improved quality of care to the US healthcare system. Rev Urol. 2017;19: Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and metaanalyses: the PRISMA statement. Ann Intern Med. 2009;151: Vickers AJ, Cronin AM, Aus G, et al. A panel of kallikrein markers can reduce unnecessary biopsy for prostate cancer: data from the European Randomized Study of Prostate Cancer Screening in Göteborg, Sweden. BMC Med. 2008;6: Nordström T, Vickers A, Assel M, et al. Comparison between the four-kallikrein panel and prostate health index for predicting prostate cancer. Eur Urol. 2015;68: Vickers A, Vertosick EA, Sjoberg DD, et al. Properties of the 4-kallikrein panel outside the diagnostic Main Points The 4Kscore Test is a commercially available, prebiopsy blood test to predict a patient's risk of high-grade prostate cancer (PCa) if a prostate biopsy were to be performed. The 4Kscore Test incorporates four kallikrein protein measurements and clinical information consisting of age, DRE result, and history of prior negative biopsy result and reports a continuous risk score from,1% to.95%. The AUC of the 4Kscore Test is above 0.80 for the discrimination of high-grade prostate cancer, and is highly consistent, across 11 clinical validation studies involving over 10,000 subjects. 154 Vol. 19 No Reviews in Urology

7 Clinical Performance of the 4Kscore Test to Predict High-grade PCa at Biopsy gray zone: meta-analysis of patients with positive digital rectal examination or prostate specific antigen 10 ng/ml and above. J Urol. 2017;197: Russo GI, Regis F, Castelli T, et al. A systematic review and meta-analysis of the diagnostic accuracy of prostate health index and 4-kallikrein panel score in predicting overall and high-grade prostate cancer. J Genitourinary Cancer. 2017;15: Voigt JD, Zappala SM, Vaughan ED, Wein AJ. The kallikrein panel for prostate cancer screening: its economic impact. Prostate. 2014;74: McDonald ML, Parson JK. 4-kallikrein test and kallikrein markers in prostate cancer screening. Urol Clin North Am. 2016; 43: Filella X, Foj L. Prostate cancer detection and prognosis: from prostate specific antigen (PSA) to exosomal biomarkers. Int J Mol Sci. 2016;17:E Lamy PJ, Allory Y, Gauchez AS, et al. Prognostic biomarkers used for localized prostate cancer management: a systematic review [published online March 7, 2017]. Eur Urol Focus. doi: /j.euf Zappala SM, Dong Y. Re: Giorgio Ivan Russo, Federica Regis, Tommaso Castelli et al. A Systematic Review and Meta-analysis of the Diagnostic Accuracy of Prostate Health Index and 4-kallikrein Panel Score in Predicting Overall and High-grade Prostate Cancer. Clinical Genitourinary Cancer 2016 [epub] DOI: Benchikh A, Savage C, Cronin A, et al. A panel of kallikrein markers can predict outcome of prostate biopsy following clinical work-up: an independent validation study from the European Randomized Study of Prostate Cancer screening, France. BMC Cancer. 2010;10: Gupta A, Roobol MJ, Savage CJ, et al. A four-kallikrein panel for the prediction of repeat prostate biopsy: data from the European Randomized Study of Prostate Cancer screening in Rotterdam, Netherlands. Br J Cancer. 2010;103: Vickers AJ, Cronin AM, Aus G, et al. Impact of recent screening on predicting the outcome of prostate cancer biopsy in men with elevated prostate-specific antigen: data from the European Randomized Study of Prostate Cancer Screening in Gothenburg, Sweden. Cancer. 2010;116: Vickers A, Cronin A, Roobol M, et al. Reducing unnecessary biopsy during prostate cancer screening using a four-kallikrein panel: an independent replication. J Clin Oncol. 2010;28: Vickers AJ, Cronin AM, Roobol MJ, et al. A fourkallikrein panel predicts prostate cancer in men with recent screening: data from the European Randomized Study of Screening for Prostate Cancer, Rotterdam. Clin Cancer Res. 2010;16: Kim EH, Andriole GL, Crawford ED, et al. Detection of high grade prostate cancer among PLCO participants using a prespecified 4-kallikrein marker panel. J Urol. 2017;197: Vol. 19 No Reviews in Urology 155