Fifteen years of molecular EQA: progress and challenges Anton M van Loon 1, Stuart West 2 and Paul Wallace 2 1 Department of Virology, UMC Utrecht, The Netherlands 2 QCMD, Glasgow, Scotland
Key Issues In Molecular Diagnostics in the Mid-90 s New technologies, rapidly moving field,, many relevant targets, high expectations Sensitivity and specificity Genotypic variation Contamination Clinical significance of (low level) nucleic acids Need for quantitative results Large number of in-house assays Lack of robustness and standardisation Lack of international reference material
False-Positivity in Molecular Diagnostics: the early years Valenthine-Thon, JCV, 2002 E Valentine-Thon et al 2001
Developments supporting improvements in molecular diagnostics 1995-2012 Anti-contamination measures: physical separation, UNG system, real-time assays Technological developments: reagents, automation, real-time platforms Introduction of commercial assays WHO International Standards: few, mainly BBV s Use of universal internal controls External quality assessment programmes
External Quality Assessment in Microbiology in Europe : end of the 1990 s EQA organisations in many countries (NEQAS, INSTAND, EQUALIS, SKMM, Labquality, etc.), but mainly focusing on (virus)culture and serology. Lack of International Standards or reference reagents for molecular testing Molecular methods: high rate of false-positives ( > 40 % ); sensitivity often unclear 1998: 3 year EU grant for EQA program MDx of (viral) neurological diseases 2001: founding of QCMD, Quality Control for Molecular Diagnostics, an independent, not-for-profit organization (www.qcmd.org), based in Glasgow
The aim of QCMD programmes is To assist laboratories to.. Evaluate the sensitivity of their assays (various genotypes/ analytical sensitivity) Determine the specificity of their assays (contamination, crossreactivity with related pathogens) Assess precision of their quantitative assays Provide an international reference where none exists: EV, HPeV, HSV, VZV, JCV, BKV, etc. Compare performance with peer laboratories Comply with regulatory requirements/ accreditation process (ISO 15189) Panels of 8-12 samples with various genotypes and microbial loads: core and educational samples
Differences in sensitivity in MDx of enteroviruses: EU-QCCA panel 1998 Virus Viral load All datasets Commercial test X (cps/ml) (n = 70) (n = 16) CA9 10 7 69 15 CA9 10 6 69 13 CA9 10 5 55 9 CA9 10 4 30 2 CA9 10 3 10 1
EVRNA11 panel : Overall Results
EV Ct value distribution: UMC Utrecht 2003-2009
Herpesvirus loads in neurological disease Aberle et al JCV 2002
HSV: Results per panel member HSV03-1 95 5 negative HSV03-2 98 2 7.6x10 6 copies/ml HSV 2 HSV03-3 62 38 6.6x10 2 copies/ml HSV 1 HSV03-4 64 36 6.6x10 2 copies/ml HSV 1 HSV03-5 97 3 1.1x10 7 copies/ml HSV 1 HSV03-6 95 5 negative HSV03-7 93 7 VZV copies/ml HSV03-8 92 8 2.0x10 3 copies/ml HSV 2 HSV03-9 77 23 2.4x10 2 copies/ml HSV 2 HSV03-10 93 7 6.1x10 3 copies/ml HSV 1 HSV03-11 93 6 1 4.8x10 3 copies/ml HSV 2 HSV03-12 83 17 4.0x10 3 copies/ml HSV 1 0% 20% 40% 60% 80% 100% % Correct % Incorrect % Equivocal
% correct results in samples from QCMD HSV and VZV programmes 2003-2013 Mean viral load (cps/ml) HSV-1 VZV 2003 2006 2009 2013 2003 2006 2009 2013 300-900 62.0% ND ND 85.5% ND 82.2% ND ND 100 300 ND 63.1% 75.8% 72.7% ND ND 84.0% 78.5% <100 ND ND ND ND ND ND 46.4% 59.9%
% correct results in samples from QCMD JCV and BKV programmes 2007 2012 Mean viral load (cps/ml) JCV BKV 2007 2010 2012 2007 2010 2012 300-600 42.7% 57.1% 82.0% 58.7% ND 96.0% 100 300 ND 64.8% * 74.8% 52.0% 72.1% ND <100 ND ND 45.0% ND 54.3% 68.3% * Clinical strain
HSVDNA 2011: % correct results vs target gene HSV Strain Load (cps/ml) DNA Pol (n=67) Target gene gb (n=56) US7/US2 (n=25) HSV-1 McIntyre 7400 98.5% 98.2% 96% 143 49.3% 53.6% 96% HSV-2 MS 5900 98.5% 100% 100% 141 55.2% 57.1% 16%
Influenza virus detection and typing 2006 A collaboration between EISS, QCMD and ESCV Subtype Dilution % correct results (n=60) Detection Typing (A, B) Subtyping (H) A, H1 10-2 97% 88% 53% A, H3 2 x 10-5 95% 85% 50% 2 x 10-6 77% 65% 28% A, H5 10-3 77% 73% 67% 2 x 10-5 48% 43% 30% A, H7 4 x 10-5 68% 53% 22% B 10-2 77% 65% - Negative - 90% -* -* * : 7.5% reported INFA; 2.5% reported INFA, H5
Influenza virus detection and typing 2012
Sam ple Sam ple Sam ple conc. content Genotypic variation-pcr False-negatives: the Swedish variant of Ct. Copies/vial datasets Commercial In-house n=211 n=35 n=4 n % n % n % n % n % n % CTA10-02 Chlamydia trachomatis 5700 207 98.1 33 94.3 4 100.0 89 97.8 28 100.0 53 100.0 CTA10-05 Chlamydia trachomatis 280 198 93.8 33 94.3 3 75.0 86 94.5 28 100.0 48 90.6 CTA10-01 Chlamydia trachomatis 280 190 90.0 31 88.6 4 100.0 84 92.3 25 89.3 46 86.8 CTA10-04 Chlamydia trachomatis 57 150 71.1 27 77.1 3 75.0 68 74.7 20 71.4 32 60.4 CTA10-03 C. trachomatis (Sw edish Variant) 2.27x10-2 Diln 178 84.4 3 8.6 3 75.0 91 100.0 28 100.0 53 100.0 CTA10-06 Ct. negative urine 207 98.1 33 94.3 3 75.0 91 100.0 27 96.4 53 100.0 CTA10-08 Chlamydia trachomatis 5700 211 100.0 35 100.0 4 100.0 91 100.0 28 100.0 53 100.0 CTA10-10 Chlamydia trachomatis 57 177 83.9 30 85.7 4 100.0 85 93.4 23 82.1 35 66.0 CTA10-07 Chlamydia trachomatis 28 163 77.3 25 71.4 3 75.0 83 91.2 22 78.6 30 56.6 CTA10-09 Ct. negative sw ab 207 98.1 34 97.1 4 100.0 90 98.9 28 100.0 51 96.2 PCR Total Conventional Real time Commercial n=91 In-house n=28 Other n=53 C. trachomatis (Swedish Variant): this strain of the pathogen is lacking 377 base pairs of the cryptic plasmid, which is commonly used as a target for molecular assays. Other: SDA, TMA. Diln: Dilution of a stock.
Genotypic variation-pcr False-negatives: the Swedish variant of Ct (2012) C. trachomatis (Swedish Variant): this strain of the pathogen is lacking 377 base pairs of the cryptic plasmid, which is commonly used as a target for molecular assays. Other: SDA, TMA.
QCMD 2010 Legionella distribution: sensitivity and contamination issues
QCMD 2012 Legionella distribution: sensitivity and contamination issues
QCMD EV/ HPeV EQA 1999 2013: False-positivity General Performance EV-A EV03 EV05 EV07 EV09 EV11 EV12 EV13 1999 2003 2005 2007 2009 2011 2012 2013 Enterovirus Participant Numbers 63 101 110 151 203 231 247 256 No Lab reporting 59 85 100 142 187 222 228 237 Total No of datasets 71 89 107 150 209 245 265 275 Participant countries 16 22 20 27 28 30 30 30 % False positives 3,6 6,5 3,7 7,5 5,7 3,0 1,1 1.5 Parechovirus Total No of datasets 23 45 67 80 81 % False positives 13,0 0,0 1,5 0,0 1.1
False-positivity rate in QCMD EQA panels 2004-2012
QCMD 2011 Aspergillus distribution: sensitivity and contamination issues
Observed VL variation in the presence WHO IS and usage of commercial assays (2008) Virus International Standard SD range of geometric mean (log10) % of commercial assays HIV Y 0.17 0.27 > 95% HCV Y 0.20 0.30 > 95% HBV Y 0.30 0.45 ~ 70% BKV N 0.5 0.6 ~ 30% HSV N 0.7 1.1 ~ 25% EV N >1.0 < 10%
Spread of commercial kit quantitative results from the mean, BBV programmes 2007-2011
Spread of in-house assay quantitative results from the mean, BBV programmes 2007-2011
Overall precision in selected QCMD distributions Virus 2008 2012 WHO Int St % IVD SD WHO % SD Median Range Int St IVD Median Range CMV N 40 0.48 0.40-0.52 Y 57 0.44 0.38-0.56 EBV N 38 0.58 0.48-0.66 Y 54 0.46 0.44-0.49 HSV N 26 0.88 0.71-1.14 N 48 0.55 0.38-1.06
Paired sample analysis QCMD CMV programmes 2010-2012 Paired sample precision in the CMVDNA EQA 2010-12 % of results 100,0 90,0 80,0 70,0 60,0 50,0 40,0 30,0 20,0 10,0 0.3-0.5 log10 units Within 0.3 log10 units 0,0 CMVDNA10 CMVDNA11 CMVDNA12
Paired sample analysis QCMD EBV programmes 2010-2012 Paired sample precision in the EBVDNA EQA 2010-12 % of results 100,0 90,0 80,0 70,0 60,0 50,0 40,0 30,0 20,0 10,0 0.3-0.5 log10 units Within 0.3 log10 units 0,0 EBVDNA10 EBVDNA11 EBVDNA12
EBV quantification QCMD 2010 proficiency programme
Technology variation: Results of the 2009 HHV6 EQA (HHV6-B) 6.000 5.500 Technology Consensus (log10 Copies/ml) 5.000 4.500 4.000 3.500 3.000 2.500 2.000 1.500 y = 0.994x + 0.1484 R² = 0.9992 y = 1.0082x - 0.288 R² = 0.9992 Real time Commercial PCR Real time In-house PCR 1.000 1.000 2.000 3.000 4.000 5.000 6.000 Consensus (log10 Copies/ml)
Use of QCMD panels as reference in assay evaluation
Summary and Conclusions Significant improvements have been achieved in MDx over the past 15 years with regard to sensitivity, specificity and precision QCMD programmes provide a reference when International Standards or reference materials are lacking A need for further improvement still exists, particularly in the non-viral area and where reference material is lacking (JCV, BKV, HHV6, ADV, respiratory/gastro pathogens) EQA programmes will also need to address new technological developments, such the increasing use of multiplex assays, Maldi-Tof and NGS