Receiver operating characteristics of the prostate specific antigen test in an unselected population

Transcription

1 102 ORIGINAL ARTICLE Receiver operating characteristics of the prostate specific antigen test in an unselected population David J McLernon, Peter T Donnan, Mike Gray, David Weller and Frank Sullivan... J Med Screen 2006;13: See end of article for authors affiliations... Correspondence to: David McLernon, Centre for General Practice, Community Health Sciences, University of Dundee, Ninewells Hospital and Medical School, Dundee DD2 4AD, UK; Accepted for publication 5 April Objectives To determine the operating characteristics of prostate-specific antigen (PSA) testing and prostate cancer diagnosis rates in men who have had an initial PSA test in. Setting A retrospective cohort study in, Scotland from 1992 to Methods In total, 20,623 men were PSA tested during the period After exclusions, 19,660 were studied. Sensitivity and specificity were calculated for various PSA cut-off values by age group using logistic regression weighted for verification bias (biopsy). Cox regression analysis was performed using six test pattern cohorts. Results The annual rate of PSA testing increased from 5.1 per 1000 man years in 1992 to 21.3 per 1000 man years in The average number of PSA tests per patient increased from 1.11 in 1992 to 2.57 in Prostate cancer diagnosis and death rates remained constant from 1995 onward. The PSA test had generally inadequate sensitivity and specificity values, so a unique cut-off could not be found for the two older age groups which could be used as a recommendation for biopsy. The commonly used 4 ng/ml cut-off was reasonably sensitive and specific only for the under 60 age group with values of 92.4% and 90.7%, respectively. For prostate cancer diagnosis, the hazard ratios (HR) were reported relative to those with a series of all normal tests. For those with an initially normal PSA test who had at least one abnormal retest result the HR for diagnosis was (95% confidence interval [CI] ). For those with initially abnormal tests with normal retests HR ¼ 1.63 (95% CI ). Conclusions There are no optimal PSA cut-off values for older age groups with which to make a confident referral for biopsy. The increase in PSA testing and the questionable cut-off values of the test calls for the development of an alternative screening strategy. INTRODUCTION Compared with many industrialized countries, the UK has a relatively low rate of testing for occult prostatic cancer using the prostate-specific antigen (PSA) test: however, recent analyses of trends in England, Wales and Northern Ireland suggest the rate is increasing. 1 3 During , National Health Service (NHS) guidance on PSA testing was issued recommending a process of informed consent rather than active discouragement of PSA testing. 4 The decision to order a PSA test remains a challenging one for clinicians because of the difficulty of interpreting the result. Clinicians order PSA tests for a variety of reasons, including as a method of investigating symptoms which may have a benign or malignant cause and in response to patient demand. 5 Prostate enlargement from a variety of conditions may result in raised PSA levels. 6 The Prostate Cancer Prevention Trial investigated the prevalence of prostate cancer among men who had a PSA level of less than 4 ng/ ml, and found that some men with levels as low as 0.5 ng/ ml throughout the study period still developed prostate cancer. 7,8 In a later study, they found that no PSA cut-off had both high sensitivity and high specificity at the same time. 9 However, a five-year follow-up study in Finland showed a higher specificity (94%) for PSA testing than for tests used to screen for breast and cervical cancer. 10,11 The values reported vary over time for example, many men with abnormal levels of PSA have normal levels in subsequent retests. 12 The ability to link anonymized Scottish cancer registry data to laboratory data provided an opportunity to study these issues in the population. Few other countries have information which combines high quality data, consistency, national coverage and the ability to link data to allow patient-based analysis and follow-up. 13,14 The aim of this study was to investigate the sensitivity and specificity of the PSA test using various cut-off points, and to follow-up initially healthy men who had one or more diagnostic PSA tests to determine the risk of prostate cancer diagnosis associated with different patterns of testing. METHODS The study population was initially derived from a laboratory database which contained all PSA test results from men in during the 10-year period from 1992 to A total of 20,623 men aged 30 and over were found to have had at least one PSA test. Data were extracted from the Health Informatics Centre (HIC) which is described in detail elsewhere. 13 The databases relevant to this study, listed below, were used within procedures approved under the data protection act and by the Caldicott Guardian. (1) Death registry: Details of all deaths in including date of death and cause of death. These were obtained from the General Register Office (GRO) for Scotland, which holds all registered deaths. Journal of Medical Screening 2006 Volume 13 Number 2

2 PSA ROC 103 (2) Carstairs categories for deprivation: The Carstairs score 15 is a socioeconomic measure based on data derived from the decennial census and linked to all residents in via the postcode. The score was divided into six categories in order of deprivation, where one is most affluent and six is the most deprived. (3) Migration: Dates of the latest immigration and emigration to and from. (4) Regional biochemistry database: Biochemical investigations since 1992 containing the PSA results and test dates. (5) Hospital pathology database: Results from all prostate pathology investigations since (6) SMR06 database: Contains all malignant neoplasms and carcinomas in situ of the prostate registered from NHS Board area residents and diagnosed during the period This information was received from Information Services Division (ISD), Scotland. (7) SMR01 database: Holds hospital admissions and procedures for all hospitals in Scotland. Prostatectomy data were obtained for the study population. The SMR06 cancer registry data were used to ascertain which men had known prostate cancer before the initial PSA test in the study period in order to exclude tests for monitoring purposes. The pathology dataset held the dates of all biopsies and if these were taken before the initial study PSA test these subjects were also excluded. All men who had a PSA test in the first six months prior to study start (1 January 30 June 1992) were also excluded from the study, to include only incident tests. In this way we could exclude those men who had a high PSA test, but were being monitored before further diagnostic tests were done. These exclusions left a nine and a half-year study period from July 1992 to December 2001 containing men who were assumed to have no clinically apparent prostate disease at least until their first PSA test in the study. Statistical methods Frequencies and the average annual rates of men PSA tested per 1000 man years (TMY) were tabulated by age group and deprivation category. Yearly rates of testing and yearly diagnosis frequencies were also calculated. Numbers of biopsies and prostate cancers diagnosed within one year of the first PSA test were tabulated by age group, deprivation category and PSA score. The sensitivity and specificity of the PSA test to diagnose cancer were calculated for various cutoff values and by age group. The sensitivity is defined as the proportion of later diagnosed men with a PSA value greater than the cut-off and the specificity as the proportion of undiagnosed men with a PSA less than or equal to the cutoff value. Since prostate cancer verification by biopsy can be subject to selection bias, the predicted probability of biopsy was found by fitting a logistic regression model with age and PSA result as predictors. 16 This probability of biopsy was then used to weight the final logistic regression model predicting the outcome of diagnosis of prostate cancer for all men and by age group. Each model had separate binary PSA cut-offs as predictors of diagnosis. Patients were categorized as diagnosed if they were diagnosed within one year of their initial PSA test. Survival analysis, by six predefined PSA test patterns, was conducted to investigate whether different patterns of normal/abnormal results using a 4 ng/ml cut-off affects the time to diagnosis in men who had PSA tests during the period The mutually exclusive PSA test patterns and their definitions are as follows: (1) All normal tests: Acts as a reference category with which to compare the other five patterns. (2) Initially normal test with at least one abnormal retest: To follow up those who were initially normal, but subsequently demonstrated an abnormal result. This category could include false positives. (3) Initially abnormal test that always retests normal: Represents false positive PSA results. (4) Initially abnormal test with no retest: Those who should have been followed up, or who were too ill or old to be investigated further. (5) Initially abnormal test that retests as normal and abnormal: A sporadic pattern which will have many interpretations. (6) Initially abnormal test that always retests abnormal: Probably represents patients with known disease who were being monitored. The starting point was taken as the date of the initial PSA test and the endpoint was 31 December 2001, prostate cancer diagnosis, emigration or death, whichever was earlier. All patients whose endpoint was not diagnosis were censored. This analysis was performed using the Cox Proportional Hazards model adjusting for age and social deprivation. The proportional hazards assumption was checked using plots of the log of the negative log of the survival function. Kaplan Meier graphs were plotted to display the survival functions by test pattern. Some basic analyses were performed using Excel, but the majority of statistical analyses and programming was conducted using the SAS (v8) software package (SAS Institute, Cary, NC, USA). RESULTS In total, 19,660 men over 30 years of age who were PSA tested for the first time between July 1992 and December 2001 were included in the analysis (Figure 1). Between them, they had 39,599 PSA tests during this study period. As would be expected, PSA testing increases with age almost linearly (Table 1). The average annual rate of healthy men PSA tested per TMY decreases from the second most affluent category to the most deprived. Almost two-thirds of men had all normal results (i.e. 13,103). From Table 2 it is evident that the rate of patients being PSA tested increased from 1992 to The number of PSA tests increased every year, as did the average number of PSA tests, which rose from 1.11 PSA tests per patient in 1992 to 2.57 PSA tests per patient in Altogether, 1114 men were diagnosed with prostate cancer during the study period. Diagnosis figures from 1995 to 2001 have remained quite steady even though more men have been tested each year. Sensitivity and specificity at different cut-off points Table 3 displays the number of men biopsied, the number of cancers verified, and the number of cancers verified strictly from biopsy. These are broken down by age group, deprivation category and a selection of PSA ranges. Quite a high number of men (237) were biopsied who had PSA values less than 4 ng/ml, of whom 14.8% did have prostate cancer. Firstly, the predicted probability of biopsy was found Journal of Medical Screening 2006 Volume 13 Number 2

3 104 McLernon et al. Initial PSA tested men (n=20,623) Known prostate cancer (n=750) 6 months pre-study (n=161) Migrated before first PSA and missing data (n=52) After removal of known prostate cancer (n=19,873) After 6 month prestudy exclusions (n=19,712) Final data (n=19,660) SMR06 only (n=459) SMR06 and pathology (n=236) Lived outside (n=47) Pathology only (n=55) Not recorded in SMR6 (n=8) Figure 1 Flowchart showing how the final number of healthy men PSA tested was found Table1 Characteristics of the population at initial PSA test and average annual rate of testing, n ¼ for each man from a logistic regression model adjusting for age, PSA score and their interaction (age by PSA score). Secondly, Table 4 contains the sensitivity and specificity scores for diagnosis resulting from further logistic regression models, each adjusted for different binary PSA cut-offs, fitted for each age group. The model was weighted by the predicted probability of biopsy. The regularly used cut-off of o4 ng/ml has a reasonable sensitivity over all age groups, but only a high specificity for the under 60s, and a lower specificity for the other two age groups (as low as 57.2% for the oldest age group). Neither of the two older age groups had a cut-off with sensitivity and specificity over 90%. For the year olds the first PSA score to have both sensitivity and specificity over 80% is 6 ng/ml, which is clearly inadequate. For the oldest age group the lowest PSA score to fulfil these criteria is 10 ng/ml. Survival analysis % population Age group , , , Total , Carstairs category 1 (affluent) , , , , , and 7 (most deprived) , Missing 13.4 Total ,966 Mean annual rate of men tested per TMY The male population figures came from the cross-sectional National Community Health Index (CHI) snapshots for 1996 TMY; Thousand man years Table 5 displays the Cox Proportional Hazards results for a model in which pattern of test results are fitted categorically to test for the relative hazard of diagnosis against test pattern one (all normal tests). The model was also adjusted for the effects of age and social deprivation. The hazard of diagnosis after an initially abnormal test with all normal retests compared to all normal tests is 1.63 (95% confidence interval [CI] ) for the PSA cut-off of o4 ng/ml. With the exception of this test pattern all other test patterns had significantly higher hazard ratios. For example, the relative hazard of being diagnosed with prostate cancer after one abnormal test with no retests is 55 times higher than those patients who always test normal. For an initially abnormal test with abnormal retests the hazard of diagnosis is (95% CI ). Those patients with an initially normal test with at least one abnormal retest have a hazard of diagnosis of (95% CI ) and for those with an initially abnormal test with normal and abnormal retests the hazard is 5.27 (95% CI ). Figure 2 displays the Kaplan Meier plot of the survival functions from initial PSA test to prostate cancer diagnosis by test pattern for the PSA cut-off value of 4 ng/ml. The plots of the log of the negative log of the survival function were approximately parallel indicating that the hazards were approximately proportional. When the all-cause mortality survival model was adjusted for the effect of prostatectomy (yes/no) the hazard ratio of 2.03 (95% CI ) was significant (P ¼ 0.009), but the test pattern hazard ratios remained the same. Therefore prostatectomy was not included in the final model. DISCUSSION Our study over a 10-year period demonstrated that PSA testing is increasing and men are being retested more often in, Scotland. Although the PSA score appears to have quite a reasonable sensitivity and specificity for those under 60, the same cannot be said for the other two age groups. No single PSA cut-off stands out as the optimum one to use. In addition, the Cox regression analysis showed that the hazard of prostate cancer diagnosis for patients with an initially abnormal PSA test with all normal retests was not significantly different from those with all normal tests. These results are derived from unselected real-world observations in a geographically defined population whose Journal of Medical Screening 2006 Volume 13 Number 2

4 PSA ROC 105 Table 2 Summary of PSA testing and rates by year Year of test Tests (n) Men (n) Mean tests per man (n) Population of men X30 years old Rate of men in tested per TMY , , , , , , , , ,526 w ,284 w Total 39,599 19, ,202, Diagnosed with prostate cancer (n) 1992 was a half-year in the study so the rates have been doubled to estimate the whole year w Population figures for 2000 and 2001 came from the cross-sectional CHI file snapshots and not from the National CHI file snapshots like the previous years. Difference is due to non- GP registered people not being counted in CHI file snapshots Table 3 Number of biopsies and cancers diagnosed by age group, PSA score and deprivation category from population (n ¼ 19660) within one year of the first PSA test Men biopsied (n) Cancers from biopsy (n) Age group (53.2) (54.9) (59.9) 600 Deprivation category 1 (lowest) (57.5) (57.7) (55.9) (61.9) (58.8) 78 6 (highest) (53.2) 105 PSA score (ng/ml) o (8.3) 7 o (7.8) 17 o (11.8) 31 o (11.8) 35 o (15.3) 53 o (16.9) 67 o (18.4) 80 o (19.4) 91 o (20.8) 105 o (22.9) 126 o (26.0) 165 o (30.4) 224 o (33.2) 278 All (59.5) 908 Cancers diagnosed (n) denominator is well defined. Their strength is the population size, the long period of follow-up and our ability to link all relevant records. 14 The limitations of retrospective data are that we have no information on the indications for the tests, medications and radiotherapy. However, we did have cancer registry data, which allowed us to exclude men with known prostate cancer before their first PSA test in the study. We also adjusted our sensitivity analysis for probability of biopsy, which reduces the effect of selection bias. The only treatment information available was prostatectomy data which, as expected, increased the hazard of diagnosis in our regression analysis but did not change the test pattern hazards. The average annual rate of PSA testing was 16.3 per TMY which is similar to the 2.0% found in England and Wales. 2 We found an association between lower testing rates and increasing social deprivation as also found elsewhere. 2 It was not clear from the sensitivity analysis which cut-off should be used as a judgement for prostate biopsy. For example, for the age group the PSA cut-offs from 6 to 10 ng/ml are all similar with inadequate sensitivity and specificity. This pattern was similar for the oldest age group. The regularly used PSA cut-off of o4 ng/ml seems to be reasonable only for the under 60 year olds. Although our sensitivity analysis did not find an optimum cut-off value to use, our sensitivity values tended to be higher with lower specificity values than found in other studies. In New Jersey, USA, a meta-analysis of PSA and DRE screening to detect prostate carcinoma found a pooled sensitivity and specificity for PSA of 72.1% and 93.2%. 17 A recent randomized prospective study by the Prostate Cancer Prevention Trial followed up men for a seven-year period after their initial PSA test and could not find a cut-off with high sensitivity and high specificity for various grades of disease and age groups (o70 and X70 years). For example, their sensitivity and specificity for 4.1 ng/ml were 20.5% and 93.8%, respectively. However, their study was confined to men with initial PSA values r3 ng/ml. 9 Our analysis showed that the PSA cut-offs with the highest sensitivity values tended to increase with age, but at the expense of poorer specificities. Age specific PSA ranges are based on the fact that PSA concentrations tend to rise with age. 18 The NHS Prostate Cancer Risk Management Programme and others support the use of age specific PSA ranges in place of the more widely used normal range of o4 ng/ml. 19,20 The fact we did not find one stand alone PSA cut-off, even by age group, which could be used to recommend biopsy suggests that there is no definite normal value of the test. The regression analysis by test pattern, which gave a nonsignificant hazard ratio for an initially abnormal test with normal retests suggested that an initial abnormal PSA result cannot be assumed to be an accurate screening marker for further, more invasive, investigations. This again, questions the value of a unique cut-off value for the test. Perhaps multiple PSA testing together with a digital rectal examination or other tests will be necessary if an accurate decision is to be made on diagnosis or further investigation. 17,21 The New Jersey study concluded that when a patient has abnormal findings using PSA and DRE, the chance of cancer is 20 25%, and when a patient has Journal of Medical Screening 2006 Volume 13 Number 2

5 106 McLernon et al. Table 4 Sensitivity and specificity for prostate cancer diagnosis by PSA score and age group All men age group age group 70+ age group PSA (ng/ml) Sensitivity Specificity Sensitivity Specificity Sensitivity Specificity Sensitivity Specificity These results are based on logistic regression models with various binary PSA cut-offs as predictors and diagnosis (yes/no) as the outcome, weighted by the predicted probability of a biopsy (based on a logistic regression model adjusted for age, PSA score and age*psa score) Table 5 Cox proportional hazards analysis from initial PSA test to diagnosis of prostate cancer by pattern of PSA testing adjusted for age and social deprivation Test pattern (vs. 1) n (%) Diagnosed (% of n) Diagnosis Hazard ratio (95% CI) (66.7) 59 (0.45) (1.9) 20 (5.41) (6.17, 17.63)*** (3.6) 6 (0.85) 1.63 (0.65, 4.07) (12.3) 639 (26.3) (41.80, 74.44)*** (4.1) 13 (1.61) 5.27 (2.75, 10.10)*** (11.4) 377 (16.76) (25.85, 46.74)*** ***Significant at the 0.1% level Test pattern key: 1 Normal tests; 2 Normal tests with at least one abnormal retest; 3 Abnormal tests that always retest normal; 4 Abnormal tests that do not retest; 5 Abnormal tests that retest as normal and abnormal; 6 Abnormal tests that always retest abnormal 1.0 Proportion diagnosed All normal tests Normal tests with at least one abnormal retest 0.2 Abnormal tests that always retest normal Abnormal tests that do not retest Abnormal tests that retest normal and abnormal Abnormal tests that always retest abnormal Weeks Figure 2 Kaplan Meier plots of time to diagnosis from first PSA test by test pattern. normal PSA and DRE findings, the chance of missing a cancer is about 10%. 17 However, a large proportion of patients with potentially curable prostate cancers can have lower PSA levels than 4 ng/ml, as test pattern 1 shows for 59 men (see Table 5) as well as other studies. 8,9 Of course increasing the number of tests will increase the likelihood of false positive results and the likelihood of a battery of tests being performed may also lead to spurious results. 22 An alternative prostate cancer screening strategy needs to be developed which will lessen unnecessary biopsies and increase necessary ones. This could mean adopting or developing a new screening test, which is cost-effective to health services. ACKNOWLEDGEMENTS The study was funded by a grant from East of Scotland Research Network (EastRen). Our thanks go to the Medicines Monitoring Unit (MEMO), a member of the MRC Health Services Research Collaboration, for the anonymisation and providing of data. Journal of Medical Screening 2006 Volume 13 Number 2

6 PSA ROC 107 We also thank Information Services Division (ISD), Scotland, for the SMR06 cancer registry data.... Authors affiliations David McLernon, Research Fellow in Medical Statistics, Centre for General Practice, Community Health Sciences, University of Dundee, Ninewells Hospital and Medical School, Dundee DD2 4AD, Scotland Peter T Donnan, Senior Lecturer in Medical Statistics, Centre for General Practice, Community Health Sciences, University of Dundee, Ninewells Hospital and Medical School, Dundee DD2 4AD, Scotland Frank Sullivan, Professor of General Practice, Centre for General Practice, Community Health Sciences, University of Dundee, Ninewells Hospital and Medical School, Dundee DD2 4AD, Scotland Mike Gray, General Practitioner, Ancrum Medical Centre, 12/14 Ancrum Road, Dundee DD2 2HZ, Scotland David Weller, Professor of General Practice, Division of Community Health Sciences, University of Edinburgh, 20 West Richmond Street, Edinburgh EH10 5PF, UK REFERENCES 1 Melia J, Moss S. Survey of the rate of PSA testing in general practice. Br J Cancer 2001;85: Melia J, Moss S, Johns L. Rates of prostate-specific antigen testing in general practice in England and Wales in asymptomatic and symptomatic patients: a cross-sectional study. BJU Int 2004;94: Gavin A, MaCarron P, Middleton RJ, et al. Evidence of prostate cancer screening in a UK region. BJU Int 2004;93: Watson E, Jenkins L, Bukach C, Austoker J. The PSA test and prostate cancer: information for primary care. Sheffield: NHS Cancer Screening Programmes, Law M. Screening without evidence of efficacy. BMJ 2004;328: Stamley TA, Caldwell M, McNeal JE, et al. The prostate specific antigen era in the United States is over for prostate cancer: what happened in the last 20 years? J Urol 2004;172: Thompson IM, Pauler DK, Goodman PJ, et al. Prevalence of prostate cancer among men with a prostate-specific antigen level o4.0 ng per milliliter. NEJM 2004;350: Roobol MJ, Kranse R, de Koning HJ, et al. Prostate-specific antigen velocity at low prostate-specific antigen levels as a screening tool for prostate cancer: results of second screening round of ERSPC (ROTTERDAM). Urology 2004;63: Thompson IM, Ankerst DP, Chi C, et al. Operating characteristics of Prostate-Specific Antigen in men with an initial PSA level of 3.0 ng/ml or lower. JAMA 2005;294: Hakama M, Stenman U-H, Aromaa A. Validity of the prostate specific antigen test for prostate cancer screening: a follow-up study with a bank of 21,000 sera in Finland. J Urol 2001;166: Hakama M, Auvinen A. Reliability and validity of prostate-specific antigen. JAMA 2003;290: Eastham JA, Riedel E, Scardino PT, et al. Variation of serum prostate-specific antigen levels: an evaluation of year-to-year fluctuations. JAMA 2003;289: Evans JM, McDevitt DG, MacDonald TM. The Medicines Monitoring Unit (MEMO): a record-linkage system for pharmacovigilance. Pharm Med 1995;9: Donnan PT, Wei L, Steinke DT, et al. Presence of bacteriuria caused by trimethoprim resistant bacteria in patients prescribed antibiotics: multilevel model with practice and individual patient data. BMJ 2004;328: Carstairs V, Morris R. Deprivation and mortality: an alternative to social class? Comm Med 1989;11: Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 1983;39: Mistry K, Cable G. Meta-analysis of prostate-specific antigen and digital rectal examination as screening tests for prostate carcinoma. JAmBdFam Pract 2003;16: Mokete M, Palmer AR, O Flynn KJ. High result in prostate specific antigen test. BMJ 2003;327: The NHS Prostate Cancer Risk Management Programme (PCRMP). accessed 05 January2005) 20 Ito K. Advancements in PSA-based screening for prostate cancer. Jpn J Clin Path 2004;52: van der Cruijsen-Koeter IW, van der Kwast TH, Schroder FH. Interval carcinomas in the European randomized study of screening for prostate cancer (ERSPC) Rotterdam. J Natl Cancer Inst 2001;95: van Walraven C, Naylor CD. Do we know what inappropriate laboratory utilisation is? A systematic review of laboratory clinical audits. JAMA 1998;280: Journal of Medical Screening 2006 Volume 13 Number 2