JCM Accepted Manuscript Posted Online 16 May 2018 J. Clin. Microbiol. doi:10.1128/jcm.00215-18 Copyright 2018 American Society for Microbiology. All Rights Reserved. 1 2 3 Analytical Comparison of the Architect Syphilis TP and Liaison Treponema Assay Automated Chemiluminescent Immunoassays and their Performance in a Reverse Syphilis Screening Algorithm 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Running Title: Syphilis TP and Treponema Assay Comparison Alan M. Sanfilippo 1,2, Kristie Freeman 1, John L. Schmitz 1,2,# 1 McLendon Clinical Laboratories, University of North Carolina Hospitals, Chapel Hill, North Carolina 2 Department of Pathology and Laboratory Medicine, University of North Carolina School of Medicine, Chapel Hill, North Carolina # Address correspondence to John L. Schmitz, John.Schmitz@unchealth.unc.edu 1
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 Syphilis incidence has increased in the United States since 2000 (1, 2) and remains a global public health concern (3, 4). A tiered serology-based testing approach, historically starting with non-treponemal testing, is recommended by The Centers for Disease Control and Prevention (CDC) (5). To increase throughput and improve sensitivity for latent syphilis, laboratories are switching to a testing algorithm that starts with an FDA cleared automated treponemal-specific chemiluminescent immunoassay (CLIA) followed by confirmation with a non-treponemal test and/or a different treponemal specific test (6-11). In this study, we compared the recently FDA-cleared Architect Syphilis TP (Abbott Laboratories, Abbott Park, IL) CLIA to the Liaison Treponema Assay (DiaSorin, Stillwater, MN) CLIA by testing 1,028 consecutive sera submitted for syphilis screening to the UNCH Clinical Immunology Laboratory. Serum was stored for no longer than 3 months at 80 0 C. All specimens were remnant specimens from UNCH patients over the age of 18 and approval was obtained from the University of North Carolina Institutional Review Board. Initial screening identified 976 nonreactive and 47 reactive specimens concordant by both methods (Table 1). Of the five discordant specimens, four were Architect reactive and Liaison nonreactive while one specimen was Liaison reactive and Architect nonreactive, providing positive, negative, and total percent agreements of 97.9%, 99.6%, and 99.5%, respectively (K = 0.947, 95% CI 0.901 to 0.993). 2
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 Comparing each assay s screening results to reverse algorithm verified positives (Supplemental Materials and Methods), the Architect Syphilis TP identified 45 of 45 verified positives, while five specimens were Architect reactive, ASI RPR (Arlington Scientific Inc., Springville UT) nonreactive, and Serodia TP-PA (Fujirebio Inc., Malvern, PA) nonreactive for a false positive rate of 0.5% (Fig. 1). No verified positive specimens were missed by the Architect, providing a sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 100% (95% CI 92.1 to 100%), 99.5% (95% CI 98.8 to 99.8%), 90.0% (95% CI 78.2 to 96.7%), and 100.0% (95% CI 99.6 to 100.0%), respectively (Table 2). The Liaison Treponema Assay also identified 45 of 45 verified positives, while two specimens were positive by the Liaison, but RPR and TP- PA nonreactive for a false positive rate of 0.2% (Fig. 1). The Liaison also had no false negative specimens for a sensitivity, specificity, PPV, and NPV of 100.0% (95% CI 92.1 to 100.0%), 99.8 % (95% CI 99.3 to 100.0%), 95.7% (95% CI 85.5 to 99.5%), and 100.0% (95% 99.6 to 100.0%), respectively (Table 2). One specimen was falsely positive on both the Architect and Liaison, while an additional specimen positive using both assays was RPR nonreactive but TP-PA indeterminate (Table 3). The inter- and intra-assay precision of both assays was comparable, with the CVs of all RX specimens being equal to or less than 3.5% (Suppl. Table 1). Considering the reverse testing algorithm involves one additional test compared to the traditional algorithm, strategies to identify potential false positives and reduce unnecessary tests have been employed by using chemiluminescent signal strengths to set a cut-off for confirmatory testing (12-14). Optimal Architect signal to cut-off (S/CO) 3
64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 and Liaison index values used to predict TP-PA confirmatory testing results were determined using receiver operating characteristic (ROC) curves. A S/CO value that provided 100% specificity was determined for the Architect Syphilis TP, as all specimens with a S/CO value greater than 6.0 (40/40; p = 0.0001) were confirmed by TP-PA testing (Fig. 2A). For the Liaison Treponema Assay, an index value greater than 3.0 provided 100% specificity as all specimens testing higher than 3.0 (45/45) were confirmed by TP-PA testing (Fig. 2B, p = 0.005). Adopting a testing approach where subsequent testing is based off of CLIA signal strengths could significantly reduce costs for a laboratory (13), however, a larger sample size which includes more CLIA reactive but RPR and TP-PA nonreactive specimens will be needed to determine reliable S/CO and Index values to use clinically. Similar studies performed outside the United States have evaluated the Architect Syphilis TP, with reported sensitivities of 99.5-100.0% and specificities of 54.5-100.0%, depending on patient population (15, 16). In conclusion, the Architect Syphilis TP exhibited similarly high sensitivity and specificity compared to the Liaison Treponema Assay and is suitable as a syphilis screening assay in a clinical laboratory setting. 4
85 Figure Legends 86 87 Fig. 1. Reverse syphilis testing algorithm. 88 89 90 91 92 93 94 Fig. 2. ROC analysis to predict true positive (TP-PA confirmed) specimens. Signal/cut-off (S/CO) and index values providing 100% specificity were selected for the Architect Syphilis TP (A; manufacturer s reactive S/CO 1.00) and Liaison Treponema Assay (B; manufacturer s equivocal index 0.90 < 1.10 and positive index 1.10) for which all specimens with higher values resulted in RX TP-PA results. p = 0.0001 for (A) and p = 0.005 for (B) using Fisher s exact test. Downloaded from http://jcm.asm.org/ on January 5, 2019 by guest 5
95 Tables 96 97 98 99 100 101 102 103 104 105 106 107 108 109 Table 1. Liaison vs Architect Concordance Liaison Architect RX NR Totals RX 47 4 51 NR 1 976 977 Totals 48 980 1028 NR, nonreactive; RX, reactive Positive and negative percent agreements = 97.9% and 99.6%, respectively Kappa = 0.947 (95% CI 0.901 to 0.993) Table 2. Reverse Algorithm Comparison Verified Positives Liaison RX (%) NR (%) Totals (%) Sensitivity (%) Specificity (%) PPV (%) NPV (%) RX (%) 45 (4.4) 2 (0.2) 47 (4.6) NR (%) 0 (0.0) 980 (95.4) 980 (95.4) Totals (%) 45 (4.4) 982 (95.6) 1027 100.0 99.8 95.7 100.0 Verified Positives Architect RX (%) NR (%) Totals (%) Sensitivity (%) Specificity (%) PPV (%) NPV (%) RX (%) 45 (4.4) 5 (0.5) 50 (4.9) NR (%) 0 (0.0) 977 (95.1) 977 (95.1) Totals (%) 45 (4.4) 982 (95.6) 1027 100.0 99.5 90.0 100.0 NPV, negative predictive value; NR, nonreactive; PPV, positive predictive value; RX, reactive 6
110 111 Table 3. Discordant Specimen Results Specimen Liaison Architect ASI RPR TP-PA Interpretation 610 NR RX NR NR Architect False Positive 948 NR RX NR NR Architect False Positive 1002 NR RX NR NR Architect False Positive 1025 RX RX NR NR Dual False Positive 1128 NR RX NR NR Architect False Positive 1282 RX NR NR NR Liaison False Positive 1493 RX RX NR IND a Undefined IND, indeterminate; NR, nonreactive; RX, reactive Downloaded from http://jcm.asm.org/ on January 5, 2019 by guest 7
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 References 1. Patton ME, Su JR, Nelson R, Weinstock H, Centers for Disease Control and Prevention. 2014. Primary and secondary syphilis--united States, 2005-2013. MMWR Morb Mortal Wkly Rep 63:402-6. 2. Centers for Disease Control and Prevention. 2016. Sexually Transmitted Disease Surveillance 2015. 3. Newman L, Rowley J, Vander Hoorn S, Wijesooriya NS, Unemo M, Low N, Stevens G, Gottlieb S, Kiarie J, Temmerman M. 2015. Global Estimates of the Prevalence and Incidence of Four Curable Sexually Transmitted Infections in 2012 Based on Systematic Review and Global Reporting. PLoS One 10:e0143304. 4. World Health Organization. 2016. Report on global sexually transmitted infection surveillance 2015, on World Health Organization, Geneva, Switzerland. http://apps.who.int/iris/bitstream/10665/249553/1/9789241565301-eng.pdf?ua=1. Accessed 20 April 2017. 5. Workowski KA, Bolan GA, Centers for Disease Control and Prevention. 2015. Sexually transmitted diseases treatment guidelines, 2015. MMWR Recomm Rep 64:1-137. 6. Binnicker MJ, Jespersen DJ, Rollins LO. 2012. Direct comparison of the traditional and reverse syphilis screening algorithms in a population with a low prevalence of syphilis. J Clin Microbiol 50:148-50. 7. Gratrix J, Plitt S, Lee BE, Ferron L, Anderson B, Verity B, Prasad E, Bunyan R, Zahariadis G, Singh AE. 2012. Impact of reverse sequence syphilis screening on 8
135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 new diagnoses of late latent syphilis in Edmonton, Canada. Sex Transm Dis 39:528-30. 8. Loeffelholz MJ, Binnicker MJ. 2012. It is time to use treponema-specific antibody screening tests for diagnosis of syphilis. J Clin Microbiol 50:2-6. 9. Mishra S, Boily MC, Ng V, Gold WL, Okura T, Shaw M, Mazzulli T, Fisman DN. 2011. The laboratory impact of changing syphilis screening from the rapidplasma reagin to a treponemal enzyme immunoassay: a case-study from the Greater Toronto Area. Sex Transm Dis 38:190-6. 10. Rhoads DD, Genzen JR, Bashleben CP, Faix JD, Ansari MQ. 2017. Prevalence of Traditional and Reverse-Algorithm Syphilis Screening in Laboratory Practice: A Survey of Participants in the College of American Pathologists Syphilis Serology Proficiency Testing Program. Arch Pathol Lab Med 141:93-97. 11. Rourk AR, Nolte FS, Litwin CM. 2016. Performance Characteristics of the Reverse Syphilis Screening Algorithm in a Population With a Moderately High Prevalence of Syphilis. Am J Clin Pathol doi:10.1093/ajcp/aqw182. 12. Dai S, Chi P, Lin Y, Zheng X, Liu W, Zhang J, Zeng Q, Wu X, Liu W, Wang J. 2014. Improved reverse screening algorithm for Treponema pallidum antibody using signal-to-cutoff ratios from chemiluminescence microparticle immunoassay. Sex Transm Dis 41:29-34. 13. Berry GJ, Loeffelholz MJ. 2016. Use of Treponemal Screening Assay Strength of Signal to Avoid Unnecessary Confirmatory Testing. Sex Transm Dis 43:737-740. 9
156 157 158 159 160 161 162 163 164 165 166 14. Yen-Lieberman B, Daniel J, Means C, Waletzky J, Daly TM. 2011. Identification of false-positive syphilis antibody results using a semiquantitative algorithm. Clin Vaccine Immunol 18:1038-40. 15. Malm K, Andersson S, Fredlund H, Norrgren H, Biague A, Mansson F, Ballard R, Unemo M. 2015. Analytical evaluation of nine serological assays for diagnosis of syphilis. J Eur Acad Dermatol Venereol 29:2369-76. 16. Marangoni A, Nardini P, Foschi C, Moroni A, D'Antuono A, Bacchi Reggiani L, Cevenini R. 2013. Evaluation of the BioPlex 2200 syphilis system as a first-line method of reverse-sequence screening for syphilis diagnosis. Clin Vaccine Immunol 20:1084-8. Downloaded from http://jcm.asm.org/ on January 5, 2019 by guest 10