Hepatitis C Virus (RNA)A 2012 EQA Programme Final Report QAV994112 (HCVRNA12A) Professor Jacques Izopet Scientific Expert on behalf of QCMD Report authorised by the QCMD Executive in July 2012 A UKAS accredited proficiency testing provider (no. 4385) Not to be reproduced or quoted without permission of QCMD. Any queries about this report should be addressed to the QCMD Neutral Office. Unit 5, Technology Terrace, Todd Campus, West of Scotland Science Park, Glasgow, G20 0XA, Scotland Tel: +44 (0)141 945 6474, Fax: +44 (0)141 945 5795, Email: info@qcmd.org, Web: www.qcmd.org
Percentage of datasets 2012 Hepatitis C Virus (RNA) A 1. Programme aims To assess the proficiency of laboratories in detection and quantitation of Hepatitis C Virus (HCV) RNA. Participants are encouraged to read the QCMD Participants' Manual, which can be downloaded from the QCMD website. Any queries about this report should be addressed to the QCMD Neutral Office (neutraloffice@qcmd.org). 2. Programme details HCVRNA12A Date of panel distribution 16/04/2012 Number of respondents 256 (94%) Number of participants 273 Number of datasets submitted 273 Number of countries 40 Number of qualitative datasets submitted 131 Number of quantitative datasets submitted 228 Seventeen participants did not return results. Three of these withdrew officially, citing 'assay not offered' (n=2) and 'technical issues' (n=1). 3. Panel composition content * matrix conc. IU/ml Log 10 IU/ml status type HCV12A-05 HCV Type 1b Plasma 18,197 4.260 Frequently detected Core HCV12A-06 HCV Type 1b Plasma 1,995 3.300 Frequently detected Core HCV12A-01 HCV Type 1b Plasma 208 2.318 Frequently detected HCV12A-07 HCV Type 3a Plasma 6,194 3.792 Frequently detected Core HCV12A-02 HCV Type 3a Plasma 687 2.837 Frequently detected Core HCV12A-03 HCV Type 3a Plasma 705 2.848 Frequently detected Core HCV12A-08 HCV Type 3a Plasma 74 1.870 Detected HCV12A-04 HCV Neg Plasma Plasma Negative Core : QCMD panel sample codes for the samples distributed to participants. content: viral content of the panel samples. matrix: material used as a matrix in preparation of the panel samples. conc.: consensus values calculated from all of the data returned by participants in the specified unit, once outliers had been removed. The values are not technology specific and should not be used by participants for method comparison or as a target for individual laboratory assessment. status: the sample status assigned to each panel sample. Please see Appendix A for more information. type: panel samples classified as core proficiency samples. *Plasma: human plasma previously screened negative for HCV RNA. Panel samples HCV12A-02 and -03 were duplicate samples. 4. Programme results 4.1. Qualitative performance on the core proficiency samples 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 94.7% 3.8% 1.5% 0.0% 0.0% 0.0% 0.0% 6/6 5/6 4/6 3/6 2/6 1/6 0/6 Number of core samples correct The QCMD EQA panels contain a range of samples, designed to look at different aspects of assay performance. Panel members are designated core proficiency samples on the basis of scientific information, clinical relevance and clinical experience (published literature and professional clinical guidelines) and, where available and appropriate, established target performance limits taken from previous QCMD EQA distributions. Laboratories are expected to correctly analyse and report the core proficiency samples in order to show acceptable proficiency. 2 of 7 www.qcmd.org
4.2. Qualitative analysis of the EQA data for all panel samples The number (percentage) of correct qualitative results are presented below. Qualitative data were returned by participants as 'detected', 'not detected' or undetermined. Undetermined results were counted as incorrect for all panel samples. QCMD organises datasets according to commercial and in-house technology groups, which are Conventional PCR, Real time PCR, NASBA, SDA, TMA and bdna. Where datasets were reported as other for a technology or kit method this was reviewed by the QCMD Neutral Office and assigned to an appropriate group where possible. Table i: Number of correct qualitative results per panel member and technology type conc. content IU/ml TMA Total Conventional Real time g datasets Commercial a In-house b Commercial c In-house d n=131 n=6 n=3 n=101 n=14 n=5 n % n % n % n % n % n % n % HCV12A-05 HCV Type 1b 18,197 131 100.0 6 100.0 3 100.0 101 100.0 14 100.0 5 100.0 2 100.0 HCV12A-06 HCV Type 1b 1,995 131 100.0 6 100.0 3 100.0 101 100.0 14 100.0 5 100.0 2 100.0 HCV12A-01 HCV Type 1b 208 129 98.5 6 100.0 3 100.0 99 98.0 14 100.0 5 100.0 2 100.0 HCV12A-07 HCV Type 3a 6,194 131 100.0 6 100.0 3 100.0 101 100.0 14 100.0 5 100.0 2 100.0 HCV12A-02 HCV Type 3a 687 129 98.5 6 100.0 1 33.3 101 100.0 14 100.0 5 100.0 2 100.0 HCV12A-03 HCV Type 3a 705 130 99.2 6 100.0 2 66.7 101 100.0 14 100.0 5 100.0 2 100.0 HCV12A-08 HCV Type 3a 74 122 93.1 5 83.3 1 33.3 98 97.0 12 85.7 5 100.0 1 50.0 HCV12A-04 HCV Neg Plasma 125 95.4 6 100.0 2 66.7 96 95.0 14 100.0 5 100.0 2 100.0 PCR Other h n=2 Table ii: Qualitative performance scores per technology type Total PCR All technologies Conventional Real time TMA g Other h status Commercial a In-house b Commercial c In-house d n=131 n=6 n=3 n=101 n=14 n=5 n=2 HCV12A-05 Frequently detected 131 0 0 0 6 0 0 0 3 0 0 0 101 0 0 0 14 0 0 0 5 0 0 0 2 0 0 0 HCV12A-06 Frequently detected 131 0 0 0 6 0 0 0 3 0 0 0 101 0 0 0 14 0 0 0 5 0 0 0 2 0 0 0 HCV12A-01 Frequently detected 129 0 0 2 6 0 0 0 3 0 0 0 99 0 0 2 14 0 0 0 5 0 0 0 2 0 0 0 HCV12A-07 Frequently detected 131 0 0 0 6 0 0 0 3 0 0 0 101 0 0 0 14 0 0 0 5 0 0 0 2 0 0 0 HCV12A-02 Frequently detected 129 0 0 2 6 0 0 0 1 0 0 2 101 0 0 0 14 0 0 0 5 0 0 0 2 0 0 0 HCV12A-03 Frequently detected 130 0 0 1 6 0 0 0 2 0 0 1 101 0 0 0 14 0 0 0 5 0 0 0 2 0 0 0 HCV12A-08 Detected 122 0 9 0 5 0 1 0 1 0 2 0 98 0 3 0 12 0 2 0 5 0 0 0 1 0 1 0 HCV12A-04 Negative 125 0 0 6 6 0 0 0 2 0 0 1 96 0 0 5 14 0 0 0 5 0 0 0 2 0 0 0 Key to Table i and ii : QCMD panel sample codes for the samples distributed to participants. content: viral content of the panel samples. conc.: consensus values calculated from all of the data returned by participants in the specified unit, once outliers had been removed. The values are not technology specific and should not be used by participants for method comparison or as a target for individual laboratory assessment. Total datasets: number and percentage of datasets reporting the correct qualitative result for each panel sample. status: the sample status assigned to each panel sample. Please see Appendix A for more information. Total. All technologies: number of datasets awarded each score (0 to 3). A breakdown of the results for all datasets is also provided based on technology type. a: Roche Amplicor HCV (Manual) (n=1), Roche COBAS Amplicor HCV Monitor (n=1), Roche COBAS Amplicor HCV Test (n=1), Roche COBAS AmpliPrep/ COBAS Amplicor HCV Test (n=3). b: Details not presented. c: Abbott RealTime HCV Assay (n=25), aj ROBOSCREEN RoboGene HCV RNA quantification kit (n=1), Fast-track Diagnostics FTD Hepatitis C (n=1), GeneProof HCV Real Time PCR kit (n=3), GFE Blut mbh Virus Screening PCR Kit (n=2), Iontek Fluorion HCV QNP 2.1 (n=1), QIAGEN artus HCV PCR Kit (RG) (n=11), QIAGEN artus HCV QS-RGQ Kit (n=3), Roche COBAS AmpliPrep/ COBAS TaqMan HCV Test (n=27), Roche COBAS TaqMan HCV High Pure System Test (n=10), Roche TaqScreen MPX Test (n=15), Sacace HCV Real-TM Qual (n=1), Sacace HCV Real-TM Quant (n=1). d: Details not presented. g: Gen-Probe Procleix Ultrio Plus assay (n=1), Novartis Procleix Ultrio HIV-1/HCV/HBV Assay (n=1), Novartis Procleix Ultrio Plus HIV-1/HCV/HBV Assay (n=2), Siemens VERSANT HCV Qualitative (TMA) (n=1). h: Siemens VERSANT HCV RNA (bdna) (n=1), Siemens VERSANT HCV Genotype Assay (LiPA) (n=1). 3 of 7 www.qcmd.org
Percentage of datasets Percentage of datasets 2012 Hepatitis C Virus (RNA) A 4.3. Quantitative performance on the paired samples Assays may differ in the relative quantitative values they report. QCMD assesses quantitative proficiency based on the difference reported between paired samples, which provides a measure of proficiency independent of assay bias. In this round of the EQA, quantitative performance was assessed using a set of paired samples containing HCV Type 1b and HCV Type 3a. Participants were expected to report to within 0.5 log10 units of the median difference generated from all quantitative data in order to show acceptable proficiency. Figure i: performance on the HCV Type 1b paired samples 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 96.5% 1.8% 0.0% 0.0% 1.8% 0 - <0.5 0.5 - <1 1 - <1.5 1.5 Log 10 units from the median difference The median difference in reported concentration between the HCV Type 1b samples (HCV12A-05 and -06) was 0.961 log10 units. Figure ii: performance on the HCV Type 3a paired samples 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 93.4% 3.1% 0.0% 0.9% 2.6% 0 - <0.5 0.5 - <1 1 - <1.5 1.5 Log 10 units from the median difference The median difference in reported concentration between the HCV Type 3a samples (HCV12A-07 and -02) was 0.940 log10 units. 4 of 7 www.qcmd.org
4.4. Quantitative analysis of the EQA data and performance scores for all panel samples Quantitative scores by technology type in comparison to the consensus mean concentration for each positive panel sample. Total PCR Consensus All technologies Conventional Real time Log 10 virus Commercial a Commercial c In-house d bdna h concentration n=228 n=2 n=212 n=11 n=3 Mean SD HCV12A-05 4.260 0.219 160 48 11 6 3 0 2 0 0 0 154 41 9 5 3 5 3 2 1 0 1 2 0 0 0 HCV12A-06 3.300 0.260 161 48 8 8 3 2 0 0 0 0 154 44 6 6 2 4 3 2 2 0 1 1 0 0 1 HCV12A-01 2.318 0.321 152 59 5 5 7 2 0 0 0 0 145 56 4 3 4 5 3 1 2 0 0 0 0 0 3 HCV12A-07 3.792 0.258 162 44 13 8 1 2 0 0 0 0 152 39 13 7 1 7 3 0 1 0 1 2 0 0 0 HCV12A-02 2.837 0.278 159 45 10 9 5 2 0 0 0 0 151 42 9 8 2 6 3 1 1 0 0 0 0 0 3 HCV12A-03 2.848 0.291 157 47 14 5 5 2 0 0 0 0 147 46 12 5 2 8 1 2 0 0 0 0 0 0 3 HCV12A-08 1.870 0.287 135 50 8 7 28 2 0 0 0 0 129 45 7 7 24 4 5 1 0 1 0 0 0 0 3 : QCMD panel sample codes for the samples distributed to participants. Consensus Log10 virus concentration: the mean quantitative value and standard deviation value for each panel sample expressed in log10 units and calculated once outlying values had been removed. Total. All technologies: number of datasets awarded each score (0 to 3). refers to datasets where an upper or lower limit of detection was reported or no value was reported (these data were excluded from the scoring). SD refers to the Standard Deviation. A breakdown of the results for all datasets is also provided based on technology type. a: Roche COBAS Amplicor HCV Monitor (n=1), Roche COBAS AmpliPrep/ COBAS Amplicor HCV Test (n=1). c: Abbott RealTime HCV Assay (n=58), aj ROBOSCREEN RoboGene HCV RNA quantification kit (n=1), Anatolia Geneworks Bosphore HCV Quantification kit (n=3), Fast-track Diagnostics FTD Hepatitis C (n=1), GeneProof HCV Real Time PCR kit (n=3), Genosens HCV Real Time PCR kit (n=1), Iontek Fluorion HCV QNP 2.1 (n=2), QIAGEN artus HCV PCR Kit (n=1), QIAGEN artus HCV PCR Kit (RG) (n=20), QIAGEN artus HCV QS-RGQ Kit (n=12), Roche COBAS AmpliPrep/ COBAS TaqMan HCV Test (n=78), Roche COBAS TaqMan HCV High Pure System Test (n=27), Roche COBAS TaqMan HCV Test (n=3), Sacace HCV Real-TM Quant (n=1), Siemens VERSANT HCV RNA (kpcr) (n=1). d: Details not presented. h: Siemens VERSANT HCV RNA (bdna) (n=3). Quantitative scores by technology type in comparison to the technology consensus mean concentration per positive panel sample. Tech. Consensus Log 10 virus Real time Commercial c concentration n=212 Mean SD HCV12A-05 4.248 0.211 152 42 9 6 3 HCV12A-06 3.290 0.255 151 47 6 6 2 HCV12A-01 2.323 0.299 139 61 4 4 4 HCV12A-07 3.784 0.257 153 37 14 7 1 HCV12A-02 2.836 0.268 150 43 8 9 2 HCV12A-03 2.846 0.287 145 48 12 5 2 HCV12A-08 1.859 0.287 130 44 7 7 24 Tech. Consensus Log 10 virus concentration Real time In-house d n=11 Mean SD HCV12A-05 4.496 0.346 7 4 0 0 0 HCV12A-06 3.528 0.365 7 2 2 0 0 HCV12A-01 2.204 0.616 7 4 0 0 0 HCV12A-07 4.057 0.283 9 1 1 0 0 HCV12A-02 2.840 0.430 8 2 1 0 0 HCV12A-03 2.870 0.371 9 1 1 0 0 HCV12A-08 2.089 0.297 7 2 1 0 1 : QCMD panel sample codes for the samples distributed to participants. Tech. Consensus Log10 virus concentration: the mean quantitative value and standard deviation value for each panel sample expressed in log10 units and calculated once outlying values had been removed. The number of datasets awarded each score (0 to 3) is then presented. refers to datasets where an upper or lower limit of detection was reported or no value was reported (these data were excluded from the scoring). SD refers to the Standard Deviation. c: Abbott RealTime HCV Assay (n=58), aj ROBOSCREEN RoboGene HCV RNA quantification kit (n=1), Anatolia Geneworks Bosphore HCV Quantification kit (n=3), Fast-track Diagnostics FTD Hepatitis C (n=1), GeneProof HCV Real Time PCR kit (n=3), Genosens HCV Real Time PCR kit (n=1), Iontek Fluorion HCV QNP 2.1 (n=2), QIAGEN artus HCV PCR Kit (n=1), QIAGEN artus HCV PCR Kit (RG) (n=20), QIAGEN artus HCV QS-RGQ Kit (n=12), Roche COBAS AmpliPrep/ COBAS TaqMan HCV Test (n=78), Roche COBAS TaqMan HCV High Pure System Test (n=27), Roche COBAS TaqMan HCV Test (n=3), Sacace HCV Real-TM Quant (n=1), Siemens VERSANT HCV RNA (kpcr) (n=1). d: Details not presented. 5 of 7 www.qcmd.org
5. Comments General Comments The number of participants in the 2012 QCMD Hepatitis C Virus (RNA) EQA Programme A was 273 a similar number to the 281 in 2011. Of the 273 datasets received in 2012, 92% (n=250) were generated using commercially-available assays, a similar percentage as 2011(91%). Commercial real time PCR kits continued to be the dominant technology group with 85% of datasets submitted, slightly higher than the 81% in 2011. Commercial TMA kits continued to be the most popular non-pcr technology. The majority of participants (83.5%) returned quantitative data to QCMD, a slightly smaller proportion than 2011 (90%) and similar to previous years. Of those who returned quantitative data 37.7% also included qualitative data in their dataset. Overall qualitative data were returned by 50% of participants. Qualitative analysis All core samples were detected by 94.7% of participants. This is an improvement on the 89.4% of the datasets that detected all core samples in the 2011. Almost all participants correctly detected all HCV positive panel samples down to a concentration of 208 IU/ml for HCV Type 1b (HCV12A-01) and 74 IU/ml for Type 3a (HCV12A-08). This performance is similar to that of the 2011 EQA programme. False positive results were submitted in 6 datasets for the negative panel sample (HCV12A-04) giving a false positivity rate of 4.6%. The false positivity rate in 2011 was 1.3% (n=2). Participants were asked to provide information on the target sequence of their assay when submitting results. Of the 131 qualitative datasets submitted 99.1% (n=105/107) were generated by assays targeting the 5' untranslated region of the HCV genome. This proportion is similar to the 98.4% of datasets that targeted this region in 2011. Analysis of the quantitative EQA data Quantitative data were returned in 228 datasets, paired analysis was performed on two HCV type 1b samples and two HCV type 3a samples. Analysis showed that over 96% of quantitative datasets were able to quantify the two HCV type 1b samples to within 0.5Log10 from the median difference, this compares to 88.8% reported on similar samples in 2011. Analysis of the two HCV type 3a samples showed that over 93% of datasets were quantified to within 0.5Log10 from the median difference, similar to 2011. The range of log10 standard deviation values for all of the quantitative data returned by participants, irrespective of technology type, was 0.219 to 0.321 which was similar to previous years (2011: 0.263 to 0.321; 2010: 0.192 to 0.303). Consensus concentration values were comparable between real time commercial and real time in-house technologies. Acknowledgements Data analysis and report generation were performed by the QCMD Neutral Office. QCMD 2012. The QCMD EQA programme samples, associated reports and data generated during this programme are intended for External Quality Assessment (EQA) and Proficiency Testing (PT) purposes only. QCMD operates according to a strict Code of Practice which is in line with ISO/IEC 17043 and associated standards. Data reported in QCMD programmes is representative of a laboratory s standard diagnostic testing protocols irrespective of the technology they use. The data provided in the reports are based on technical information provided by the individual laboratories as part of the assessment process, as such it does not constitute a formal technology method comparison. All text and images produced by QCMD are the property of QCMD unless otherwise stated. The reproduction and use of these materials is not permitted without the express written consent of QCMD. The use of the information provided in QCMD reports for commercial purposes is strictly prohibited. 6 of 7 www.qcmd.org
Appendix A Assigning the sample status status is assigned by peer-group consensus, based on the qualitative results returned by all participants. It is not a measure of the 'strength' of a positive sample nor is it technology-dependent, and is used solely for the scoring of the EQA data. The rationale for the sample status is: Frequently detected: More than 95% of datasets recorded the correct positive result. Detected: Between 65% and 95% of datasets recorded the correct positive result. Infrequently detected: Less than 65% of datasets recorded the correct positive result. Negative: A panel sample that does not contain the target and produces an unequivocal negative result. Scoring system for qualitative EQA data The scores awarded for qualitative EQA data were based on the sample status. The scoring system is represented in the following table, where 0 is 'highly satisfactory' and 3 is 'highly unsatisfactory'. Colour has been included as an extra visual aid. Scoring system based on the assigned sample status status Participant's result Negative Not determined Positive Frequently detected 3 3 0 Detected 2 2 0 Infrequently detected 1 1 0 Negative 0 3 3 Calculation of the consensus and technology concentrations In order to compare participants results within specific technologies or kit methods, where sufficient datasets were reported (5 or more) methods or kits were assigned to a technology group e.g. Real time PCR, bdna or NASBA etc. Where datasets were reported as 'other' for a technology or kit method this was reviewed by the QCMD Neutral Office and assigned to an appropriate group where possible. The negative panel samples were not included in these analyses. An assigned value was calculated for each panel sample using two methods. These were: 1. Consensus concentration - the mean of the participants' results once outliers had been removed. 2. Technology consensus concentration - the mean of the participants' results per technology group once outliers had been removed. The standard deviation was calculated as the square root of the mean square for error from the ANOVA table where the response was the log concentration of participants results with outliers removed. The factor was technology group. Outliers were defined as values with a standardised residual with modulus greater than three. Although outliers were removed for the calculation of the assigned values they were included in the data analysis. Scores were awarded based on the distance from the calculated mean value for each panel sample. Zero points was awarded if the quantitative value returned was within one standard deviation from the mean. One point was awarded if the quantitative value was between one and two standard deviations, two points if the value was within two and three standard deviations and three points for quantitative values more than three standard deviations from the mean. Further information about the QCMD method of analysing EQA data can be found in the following peer-reviewed publication: Staines HJ, Garcia-Fernandez L, Pogothata R, Wallace PS, MacKay WG, van Loon AM. Monitoring performance of nucleic acid-based diagnostic measurement system users by EQA. Accred Qual Assur. 2009; 14: 243 252. 7 of 7 www.qcmd.org