The use of QCMD proficiency testing panels in clinical virology. Prof.dr. Bert Niesters Department of Medical Microbiology Division of Clinical Virology UMC Groningen
Disease Disease Management Viral Load
Under requirements of ISO 15189 ISO 15189 (Medical laboratories Particular requirements for quality and competence) 5.6.4: The laboratory shall participate in inter-laboratory comparisons. The laboratory shall monitor the results. Inter-laboratory comparison programs shall be in substantial agreement with ISO/IEC Guide 43-1.
QA & QC within the Clinical Diagnostic Laboratory New method Establish Performance Assay Verification Assay Validation Verify method Method Equivalence Characterised Quality reagents Certification Accreditation Validate/verify Performance Quality Improvements Quality Management System Assay Implementation Implement method Method Maintenance Perform Clinical tests Internal QC Report Results EQA (PT) Internal QA
What do I need for my laboratory Get birds eye view about my own performance With statistical analysis of results Composition determined by scientific experts working in my field Have a challenging component Include different serotypes / genotypes Have information of performance of related technologies Short reporting time 4-6 weeks Access to scientific experts Opportunity for lab pairing and mentoring
Feedback to me as laboratory director
EQA Individual report Aim is to provide personalised feedback to each participant My results and scores Ability to gauge my performance in relation to the rest of the participants in the final report
QCMD EQA scoring system qualitative Sam ple status Participant's result Negative Not determ ined Positive Frequently detected 3 3 0 Detected 2 2 0 Infrequently detected 1 1 0 Negative 0 3 3 The scores awarded for qualitative data are based on the sample status where 0 is 'highly satisfactory' and 3 is 'highly unsatisfactory'. Colour has been included as an extra visual aid.
QCMD EQA scoring system quantitative Based on distance from the consensus (log 10 mean) Two consensuses overall and by technology type 0 points = up to one sd 1 point = one to two sd 2 points = two to three sd 3 2 1 0 1 2 3 3 points = three or more sd
Qualitative data scoring Core sample Core sample Core sample Core sample Detected Core sample Core sample Detected Core sample Negative
Qualitative data scoring
Quantitative data scoring
Quantitative data scoring
What are blood borne pathogens? The potential standards hit list Classical targets: HIV-1*** Hepatitis B virus** Hepatitis C virus*** Cytomegalovirus* Epstein-Barr virus B19 virus* Clinical relevant targets: BK (JC) virus HSV type 1 HHV type 6 Adenovirus Hepatitis A virus* Hepatitis E virus, genotype 3 Genotyping panels for different targets (HIV-1, HBV, HCV to start with) *WHO standards available
Units used for reporting HCV, HIV-1 and HBV viral load results in EQA distributions 2002 100% Geq/ml Meg/ml 90% 80% pg/ml IU/ml Geq/ml 70% 60% 50% IU/ml HCV HIV HBV 40% Copies/ml Copies/ml 30% 20% 10% Copies/ml 0% 2002 2002 2002
Units used for reporting HCV, HIV-1 and HBV viral load results in EQA distributions 2002 & 2008 100% Geq/ml Meg/ml 90% 80% pg/ml IU/ml Geq/ml 70% 60% 50% IU/ml HCV HIV HBV 40% Copies/ml Copies/ml 30% 20% 10% Copies/ml 0% 2002 2008 2002 2008 2002 2008
1.00 Maximum & Minimum SD ranges reported within the BBV EQA programmes since 2005 0.80 0.27 0.60 0.40 0.18 0.23 0.10 HBV 0.20 0.11 0.10 HCV HIV 0.00 2005 2006 2007 2008
Need for Reference Materials: Precision of Molecular Assays Virus International Standard SD range of geometric mean (log10) % of commercial assays HIV Y 0.17 0.27 > 95% HCV Y 0.20 0.30 > 95% HBV Y 0.30 0.45 ~ 50% BKV N 0.5 0.6 ~ 30% HSV N 0.6 0.7 < 15% EV N >1.0 < 10%
Percentage positive results Strain variation 120 100 y = 38.19x - 6.314 R² = 0.955 80 60 40 y = 39.48x - 17.93 R² = 0.954 Type A/G Type C Linear (Type A/G) Linear (Type C) 20 0 1.7 1.9 2.1 2.3 2.5 2.7 2.9 3.1 Log10 Copies/ml
Technology Consensus (Log10 Copies/ml) Technology variation Results of the 2009 HSV EQA (HSV-1) 5.000 4.500 4.000 3.500 3.000 2.500 Real time Commercial PCR y = 0.9768x + 0.0628 R² = 0.9983 Real time In-house PCR y = 0.9452x + 0.1409 R² = 0.9931 2.000 1.500 1.500 2.000 2.500 3.000 3.500 4.000 4.500 Consensus (log10 Copies/ml)
Technology Consensus (log10 Copies/ml) Technology variation Results of the 2009 HHV6 EQA (HHV6-B) 6.000 5.500 5.000 4.500 4.000 3.500 3.000 2.500 y = 0.994x + 0.1484 R² = 0.9992 Real time Commercial PCR Real time In-house PCR 2.000 1.500 y = 1.0082x - 0.288 R² = 0.9992 1.000 1.000 2.000 3.000 4.000 5.000 6.000 Consensus (log10 Copies/ml) The lines predicting the Technology Consensus values have significantly different intercepts (difference = 0.436, 95% CI 0.305 to 0.568; p = 0.00002).
Quantitative accuracy/variation There is no such thing as a true gold standard in molecular diagnostics WHO supporting use of standardised international units (e.g for HIV-1) Still there is a (too) large variation on reported quantitative results
Quality Assurance in the clinical laboratory Internal Quality Assessment (IQA) Internal Quality Control (IQC) External Quality Assessment (EQA) Accreditation Quality Management system (QMS) Staff training (CPD) Certification Equipment & Process Monitoring Internal & External Audit Foundations in clinical chemistry: Belk and Sunderman 1950s
Quality assurance in molecular diagnostics: Conclusions Rapidly moving field, new technologies, new assays, high expectations In some cases the technology still has to prove itself in the clinical setting External and internal quality assessment improves molecular diagnostic methods. What is appropriate for one lab may not be for another. Education and training in quality tools is essential, but out of the scope of EQA.
How can Proficiency Testing EQA support laboratory quality? Monitor the performance of laboratory testing process. Allow comparison to other laboratories. Assist in identification of potential testing problems. Performance of multiple labs, helps identify quality issues that were not noticed in individual labs. Complements the clinical laboratory s Quality Assurance System (regulatory requirements). Cover clinically relevant targets and developing targets (including genetic variations). Educate participants in quality assurance issues.