Intra-Assay Performance Characteristics of Five Assays for Quantification of Human Immunodeficiency Virus Type 1 RNA in Plasma

Transcription

1 JOURNAL OF CLINICAL MICROBIOLOGY, Mar. 1998, p Vol. 36, No /98/$ Copyright 1998, American Society for Microbiology Intra-Assay Performance Characteristics of Five Assays for Quantification of Human Immunodeficiency Virus Type 1 RNA in Plasma HSIANG JU LIN, 1 LOUISE PEDNEAULT, 2 AND F. BLAINE HOLLINGER 1 * Division of Molecular Virology, Baylor College of Medicine, Houston, Texas, 1 and Antiviral Clinical Research Department, Bristol-Myers Squibb Pharmaceutical Research Institute, Wallingford, Connecticut 2 Received 23 June 1997/Returned for modification 7 November 1997/Accepted 1 December 1997 Three kits ( human immunodeficiency virus type 1 [HIV-1] Monitor, Chiron enhancedsensitivity bdna, and Organon Teknika NASBA HIV-1 QT) and two in-house assays (from National Genetics Institute and Baylor College of Medicine) were compared with a blinded panel. The results were evaluated as to intra-assay sensitivity, precision, and ability to detect differences in a dilution series. The quantification in plasma of human immunodeficiency virus type 1 (HIV-1) RNA is a useful test for predicting clinical outcome, evaluating disease progression, and monitoring the effect of antiretroviral treatment on HIV-1-infected patients (6, 9). The primary reason for undertaking this project was to evaluate the performance of selected methodologies or laboratories for monitoring the effect of antiretroviral therapy on HIV-1 viral load in clinical research. Three manufacturers of HIV-1 RNA kits and an independent laboratory agreed to participate in a single round of proficiency testing on a panel of Item Standards (Source: kit or in-house) Sample volume; sample preparation Amplification Chiron ES-bDNA External standards (kit): singlestranded DNA encoding gag and pol genes 1.0 ml, in duplicate; sedimentation of HIV-1 at 23,500 g/1 h, protease K digestion Signal amplification by means of hybridization of target region to probe, preamplifier, and bdna molecules TABLE 1. Methods employed for quantification of HIV-1 RNA NGI (in-house) External standards (inhouse): dilution series prepared from pooled HIV-1-positive sera and plasma 200 l; extraction with guanidinium thiocyanate-phenol-chloroform, alcohol precipitation cdna synthesis with random hexamers, PCR with four different cycle numbers with primers SK38 and SK39 or SK145 and SK431 plasma specimens. The laboratories performing the tests were chosen by the manufacturers. Enhanced-sensitivity bdna assays (ES-bDNA) were performed at the Chiron Reference Testing Laboratory (Emeryville, Calif.), tests were carried out at Novum, Inc. (Pittsburgh, Pa.), and NASBA (nucleic acid sequence-based amplification) HIV-1 QT assays were performed at Advanced Bioscience Laboratories, Inc. (Kensington, Md.). National Genetics Institute, or NGI (Culver City, Calif.), an established independent laboratory, used its in-house method. Because the panel samples were pre- Parameters for: Internal standard (kit): poly(a)- linked RNA containing conserved and rearranged sequences within the gag region 200 l; precipitation from guanidinium thiocyanate reagent, isopropanol wash Coamplification of standards and sample HIV-1 RNA by means of RT-PCR with SK462 and SK431 Organon Teknika NASBA HIV-1 QT Internal standards (kit): 1,500-nucleotide RNA homologous to the gag/ pol region, modified over a 20-nucleotide sequence 100 l; lysis in guanidinium thiocyanate reagent, adsorption to activated silica Coamplification of standards and sample by means of reverse transcriptase-, T7 polymerase- and RNase H-mediated reactions with primers conjugated to T7 promoter BCM (in-house) External standards (in-house): dilution series prepared from plasma spiked with strain HXB3 virions 140 l; spin column preparation with a commercial kit RT-PCR with primers SK38 and 5 biotinylated SK39 Detection Chemiluminescence Scanning densitometry Spectrophotometry Electrochemiluminescence Chemiluminescence * Corresponding author. Mailing address: Division of Molecular Virology, Baylor College of Medicine, 1 Baylor Plaza, Houston, TX Phone: (713) Fax: (713) blaineh@bcm.tmc.edu. Present address: Glaxo Wellcome, Inc., Research Triangle Park, NC

2 836 NOTES J. CLIN. MICROBIOL. TABLE 2. Quantification of HIV-1 RNA in 32 plasma specimens by five different methods Nominal no. of estimated HIV-1 RNA copies/ml a Chiron ESbDNA NGI (in-house) No. of HIV-1 RNA copies/ml estimated by b : BCM (in-house) Organon Teknika NASBA HIV-1 QT 2,000,000 ( 1,600,000) 1,300,000 2,050,000 (958,023) (19,000,000) 2,000,000 ( 1,600,000) 1,100,000 2,450,000 (882,538) 8,800,000 1,000,000 1,050, ,000 1,860,000 (836,158) 4,900,000 1,000,000 1,172, ,000 1,350, ,264 5,300, , , , , ,052 4,400, , , , ,000 (1,291,354) 3,700, , ,500 85, , ,356 1,400, , , , , , ,000 65,000 82,060 75,000 69, ,522 1,100,000 65,000 79,950 75, ,000 60, ,000 65,000 81,260 45,000 42,500 79, ,000 13,000 19,200 10,000 5,475 16, ,000 13,000 14,360 10,000 17,250 55, ,000 13,000 10,950 9,000 13,300 48, ,000 6,500 5,260 6,000 8,350 7,498 66,000 6,500 6,030 6,500 5,425 10,188 91,000 6,500 6,022 6,000 9,100 8,187 1,000,000 1, ,500 1,630 5,982 5,300 1,300 1, ,000 1,321 13,000 1, ,400 1,631 11, ,162 7, ( 500) ,911 8, ( 500) 500 1,200 (206) 12, ( 500) 300 ( 500) 421 ( 4,000) 130 ( 500) 150 ( 500) (123) ( 4,000) 130 ( 500) 200 ( 500) (221) ( 4,000) 65 ( 500) 200 ( 500) (341) ( 4,000) 65 ( 500) 150 ( 500) (188) ( 4,000) a As defined by standards produced at BCM. b The dynamic ranges of the assays claimed by the manufacturers or laboratories were as follows (in copies per milliliter): for ES-bDNA, 500 to ; for NGI, 100 to ; for BCM, 500 to ; for Roche, 400 to ; for NASBA, 4,000 to Reported results that fell above or below the claimed dynamic range are in parentheses. pared at the Center for AIDS Research Virology Core Facility at Baylor College of Medicine (BCM), we have also included in this report the results obtained with the in-house method that was used to establish their nominal concentrations. Summarized in Table 1 are the differences among the five assays. The ES-bDNA assay for HIV-1 RNA is a secondgeneration assay developed by the Chiron Corporation (Emeryville, Calif.) and incorporates changes in probe design and reagent composition (3). The distinctive feature of the NGI in-house assay was the use of four different numbers of PCR cycles (1). The AMPLICOR kit, manufactured by Roche Molecular Systems (Branchburg, N.J.), employed Thermus thermophilus polymerase, a recombinant enzyme capable of carrying out both reverse transcription (RT) and PCR (7). Assays based on NASBA technology developed by Organon Teknika (Durham, N.C.) were carried out under isothermal conditions and resulted in an RNA amplification product (11). The BCM method employed the Qiagen viral RNA kit (Santa Clarita, Calif.) for sample preparation and chemiluminescence assay of PCR products (Digene Diagnostics, Inc., Silver Spring, Md.) (5). HIV-1 standards and panel samples were prepared with normal acid-citrate-dextrose (ACD) plasma from individual blood donors who tested negative for anti-hiv-1, anti-hiv-2, anti-hepatitis B core, hepatitis B surface antigen, anti-hepatitis C virus, and anti-human T-cell leukemia virus type 1 (The Gulf Coast Regional Blood Center, Houston, Tex.). The primary standard was prepared by spiking negative plasma with supernatant fluid from a coculture of peripheral blood mononuclear cells from an HIV-1-infected patient (4). The secondary standard and panel samples were prepared by spiking negative plasma with HIV-1 strain HXB3 (5). Calibration of the secondary standards against the primary standard was based on 10 observations performed in five separate experiments with a between-runs coefficient of variation (CV) of 11.8%. The panel consisted of duplicate samples at , , 500,000, 130,000, and 65 HIV-1 RNA copies per ml; triplicate samples at 65,000, 13,000, 6,500, 1,300, 650, and 130 HIV-1 RNA copies per ml; and four samples of negative plasma. Aliquoting of samples was performed under optimal conditions to ensure sample uniformity. Specimens were shipped to testing sites on dry ice, and testing was performed in a blinded fashion at all sites, including the BCM laboratory. Presented in Table 2 are the results of HIV-1 RNA quantification for each sample as reported by the testing sites, compared with the nominal HIV-1 RNA concentrations. In comparisons of those results that fell within the dynamic range claimed for each method (listed in a footnote to the table), the differences between the means of results for replicate samples

3 VOL. 36, 1998 NOTES 837 FIG. 1. Linearity of HIV-1 RNA estimates performed on serially diluted spiked specimens by different assay methods. Assays employed were ES-bDNA, NGI, BCM, AMPLICOR, and NASBA HIV-1 QT. The straight diagonal line represents the equivalence of nominal and estimated HIV-1 RNA concentrations. Estimated concentrations for the dilution series that were obtained for a given assay are shown as connected points. Each point represents the mean of two or three estimates. If one or more of the reported results for a point fell outside the dynamic range of the assay, that point was omitted. Downloaded from that were obtained with the Chiron, NGI, BCM, and Roche assays generally fell within the same order of magnitude. The data in Table 2 were evaluated with respect to the ability of the assays to detect the known differences in HIV-1 RNA concentration present in the dilution series (Fig. 1). Nominal and estimated HIV-1 RNA concentrations were expressed as logarithms to the base 10. For each method, the mean of the results for each group of two or three panel samples with the same nominal HIV-1 concentration was calculated and represented as a point in the appropriate panel in Fig. 1. Groups that contained results falling outside the claimed dynamic range were omitted. For example, the ES-bDNA assay results for the three samples with 500 nominal HIV-1 RNA copies per ml and two samples with nominal copy numbers of were omitted. If an assay method correctly estimated the relationship of the various samples in the dilution series, a loglog plot of the estimated mean HIV-1 RNA concentrations against the nominal concentrations should produce a straight line having the same slope as the line of equivalence, shown in each panel as a diagonal straight line. The data set exhibiting the closest approximation of a straight line parallel to the line of equivalence was obtained by ES-bDNA. With perfect accuracy, the line representing the assay results should fall on the line of equivalence. This criterion must be applied with discretion, since the nominal concentrations of the panel samples are at best only estimates of an unknown value, the true HIV-1 RNA concentration. The results obtained with the Chiron, NGI, BCM, and Roche assays fell closest to the line of equivalence. These four assays provided estimates of HIV-1 RNA concentrations that generally agreed with each other and with the nominal concentrations of the BCM standards. In this sense, these assays were accurate. The slope of the line for the NASBA test also paralleled the line of equivalence but was displaced from it by approximately one log 10. The performance characteristics of the five assays are summarized in Table 3. The calculations of correlation, CV, and standard deviation (SD) were based solely on those results that fell within the claimed dynamic ranges. Correlation between the observed and nominal concentrations was close to for all five assays, although the results obtained by the NASBA assay were consistently higher. Intra-assay precision was expressed by the CV and as the logarithm of the SD to the base 10 (log 10 SD), a calculation that conveniently allowed comparison of SDs when the range of values covered several orders of magnitude. The ES-bDNA assay showed the smallest mean CV. An intra-assay SD of 0.15 to 0.20 log 10 RNA copies per ml is required in order to achieve the degree of precision that will provide 90% power to detect a fivefold difference in RNA concentration between two samples (12). The ES-bDNA, NGI, and BCM assays achieved this degree of precision. The duplicate log 10 SD was lower than the triplicate log 10 SD for all methods. This result was attributable to the composition of the panel. Samples tested in duplicate were preponderantly those with high HIV-1 RNA concentrations, while samples tested in triplicate were those with lower concentrations. Of the five assays, the ES-bDNA assay showed the highest overall intraassay precision, with a log 10 SD of to HIV-1-negative samples were correctly identified in all five assays, and no false positives were found; thus, the specificity was 100% for all five methods. It was demonstrated in the on September 17, 2018 by guest

4 838 NOTES J. CLIN. MICROBIOL. TABLE 3. Intra-assay performance characteristics of five methods for quantification of plasma HIV-1 RNA Performance characteristic Chiron ES-bDNA NGI (in-house) BCM (in-house) Correlation with nominal concentrations (no. of pairs) Organon Teknika NASBA HIV-1 QT (19) (28) (23) (16) (21) Mean CV (%) CV (range) % % % % % Log 10 SD: Duplicate Triplicate False-positive results (no. of results/total) experiment shown in Fig. 1 that there was little or no difference between the nominal HIV-1 RNA concentrations and the estimates of HIV-1 RNA concentration obtained by the Chiron, NGI, BCM, and Roche methods. It was thus possible to compare the dynamic range claimed for each of these four methods to the observed lower and upper limits of detection, abbreviated L and U, respectively. The observed L for ES-bDNA was higher than the claimed L; the observed H was consistent with the claimed H. NGI s observed L and U exceeded its claimed dynamic range, while BCM s observed L and U were consistent with its claims. The observed L and U of the AMPLICOR were both lower than those claimed by the manufacturer. This study underscores the importance of the use of welldefined standards for assigning nominal HIV-1 RNA concentrations to panel materials. In a previous comparison of the AMPLICOR and NASBA assays, both methods produced estimates below the nominal copy numbers (10). Compared to Monitor, NASBA has yielded higher estimates of HIV-1 RNA concentrations in two of three studies (2, 8, 10). This article describes the first performance comparison of ES-bDNA with other assays for HIV-1 RNA. Among the five laboratories that used ES-bDNA, the NGI assay, the BCM assay, Roche, and NASBA HIV-1 QT, the laboratories performing the first four tests named provided results that were generally consistent with each other. However, interlaboratory variation with the same kit is known to occur and may be greater than that which occurs with different kits (12). In other evaluations, the NASBA assay has performed well (2, 8, 10). The reason for the approximately 10- fold-higher values provided by the NASBA assay in the present study is being investigated. This event appears to be isolated, since many laboratories have successfully adopted the NASBA method (12). In our opinion, the laboratory may be more important than the assay method in the selection of a testing site for clinical trials. We thank the following persons for their cooperation: John Todd 0/4 0/4 0/4 0/4 0/4 Lower limit of detection (L) a : Claimed ,000 Observed 650 L 1, L Not evaluated Upper limit of detection (U) a : Claimed Observed U U U U Not evaluated a HIV-1 RNA copies/ml. and Pam Johnson, Chiron Corporation; Jay Weiss, Andrew J. Conrad, and Lawrence M. Blatt, NGI; Lisa Cosentino, Novum, Inc.; Beverly Dale, Roche Molecular Systems; Joe Romano, Advanced BioScience Laboratories, Inc.; and Michael Cronin, Stuart P. Geiger, and Richard J. Carroll, Organon Teknika Corp. The primary standard for HIV-1 RNA was prepared in the Eugene B. Casey Hepatitis & HIV Research Center and Diagnostic Laboratory at BCM in connection with a study (4) supported by a National Institutes of Health grant (AI-82517). Additional support was provided by the Center for AIDS Research (CFAR) Virology Core Facility grant (P30 A /05) awarded to BCM. REFERENCES 1. Conrad, A. J., P. Schmid, K. Syndulko, E. J. Singer, R. M. Nagra, J. J. Russell, and W. W. Tourtellotte Quantifying HIV-1 RNA using the polymerase chain reaction on cerebrospinal fluid and serum of seropositive individuals with and without neurologic abnormalities. J. Acquired Immune Defic. Syndr. Hum. Retrovirol. 10: Coste, J., B. Montes, J. Reynes, M. Peeters, C. Segarra, J.-P. Vendrell, E. Delapporte, and M. Segondy Comparative evaluation of three assays for the quantitation of human immunodeficiency virus type 1 RNA in plasma. J. Med. Virol. 50: Kern, D., M. Collins, T. Fultz, J. Detmer, S. Hamren, J. K. Peterkin, P. Sheridan, M. Urdea, R. White, T. Yeghiazarian, and J. Todd An enhanced-sensitivity branched-dna assay for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 34: Lin, H. J., L. E. Myers, B. Yen-Lieberman, F. B. Hollinger, D. Henrard, C. J. Hooper, R. Kokka, S. Kwok, S. Rasheed, M. Vahey, M. A. Winters, L. J. McQuay, P. L. Nara, P. Reichelderfer, R. W. Coombs, and J. B. Jackson Multicenter evaluation of quantification methods for plasma human immunodeficiency virus type 1 RNA. J. Infect. Dis. 170: Lin, H. J., T. Tanwandee, and F. B. Hollinger Improved methods for quantification of human immunodeficiency virus type 1 (HIV-1) RNA and hepatitis C virus (HCV) RNA in blood using spin column technology and chemiluminescent assays of PCR products. J. Med. Virol. 51: Mellors, J. W., C. R. Rinaldo, Jr., P. Gupta, R. M. White, J. A. Todd, and L. A. Kingsley Prognosis in HIV-1 infection predicted by quantity of virus in plasma. Science 272: Mulder, J., N. McKinney, C. Christopherson, J. Sninsky, L. Greenfield, and S. Kwok Rapid and simple PCR assay for quantitation of human immunodeficiency virus type 1 RNA in plasma: application to acute retroviral infection. J. Clin. Microbiol. 32: Revets, H., D. Marissens, S. De Wit, P. Lacor, N. Clumeck, S. Lauwers, and G. Zissis Comparative evaluation of NASBA HIV-1 RNA QT, AMPLICOR-HIV Monitor, and QUANTIPLEX HIV RNA assay, three

5 VOL. 36, 1998 NOTES 839 methods for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 34: Saag, M. S., M. Holodniy, D. R. Kuritzkes, W. A. O Brien, R. Coombs, M. E. Poscher, D. M. Jacobsen, G. M. Shaw, D. D. Richman, and P. A. Volderbing HIV viral load markers in clinical practice. Nat. Med. 2: Schuurman, R., D. Descamps, G. van Weverling, S. Kaye, J. Tijnagel, I. Williams, R. van Leeuwen, R. Tedder, C. A. Boucher, F. Brun-Vezinet, and C. Loveday Multicenter comparison of three commercial methods for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 34: van Gemen, B., T. Kievits, R. Schukkink, D. van Strijp, L. T. Matek, R. Sooknanan, H. G. Huisman, and P. Lens Quantification of HIV-1 RNA in plasma using NASBA during HIV-1 primary infection. J. Virol. Methods 43: Yen-Lieberman, B., D. Brambilla, B. Jackson, J. Bremer, R. Coombs, M. Cronin, S. Herman, D. Katzenstein, S. Leung, H. J. Lin, P. Palumbo, S. Rasheed, J. Todd, M. Vahey, and P. Reichelderfer Evaluation of a quality assurance program for quantitation of human immunodeficiency virus type 1 RNA in plasma by the AIDS Clinical Trials Group Virology Laboratories. J. Clin. Microbiol. 34: