Practical Aspects of Standardisation for a Global Controls Manufacturer MRSA SoGAT Clinical Diagnostic Meeting NIBSC, UK June 25, 2008 Frank Opdam PhD, AcroMetrix
Standardisation A fundamental goal of laboratory medicine is that results for patients samples will be comparable independent of the medical laboratory that produced the results.* *W. Greg Miller et al Clinical Chemistry. 2006;52:553-554
CMV monitoring in Canada A practical example of difficulties caused by a lack of standardisation
Probability of CMV Disease Probability of acquiring CMV disease with increasing viral load (renal patients) CMV Load (Log 10 copies/ml) *Aitken et al. J. Clin. Micro. 1999 37: 2804.
Cytomegalovirus Quantification Agreement plot for 100 plasma samples with detectable viral loads by the Abbott and Monitor tests. Dotted line, mean difference for the samples; dashed lines, 95% limits of agreement. The clinical utility of CMV viral load testing is well established and is considered the standard of care in monitoring the response to antiviral therapy. However, the lack of standardization among CMV viral load assays continues to be problematic for clinical laboratories.* *A.M. Caliendo et al. J Clin Microbiol. 2007 Jun; 45(6): 1723-7 Emory University School of Medicine, Atlanta
CMV CAP Survey College of American Pathologists (CAP) proficiency testing results shows majority of participating labs rely on laboratory developed quantitative CMV tests 2006 ID A survey
Cut-off values associated with active disease vary by specific assay type Though these tests have proven to be very useful for individual laboratories, it has been difficult to establish broadly applicable, clinically relevant guidelines for the use of laboratorydeveloped CMV viral load assays. * *Debra G.B. Leonard, Molecular Pathology in Clinical Practice (2007), Chapter 38 p. 427 Viral Infections in Transplant Patients by Andrea Ferreira-Gonzalez and Angela M. Caliendo
CMV in Canada Example of a problem Province A 1e5cp/ml Transplantation Patient moves to Province B Provincial Labs are aware of these differences and educate physicians accordingly Province B 1e4 cp/ml
CMV copies Result is 8000 CMV cp/ml for a Solid Organ Donor Transplantation What should the physician do? Depends on assay: Roche AMPLICOR CMV Intervention Point: 5000 cp/ml Qiagen Artus ASR* CMV Intervention Point: 25000 cp/ml * Analyte Specific Reagents
QCMD CMV proficiency programme Detection of clinical important viral load levels as low as 1000 copies/ml are not yet optimal and laboratories should aim for improvement October 2007 (CMVDNA07) final report
CMV Conclusion Lack of defined units and calibration material results in difficult to establish broadly applicable, clinically relevant guidelines
Assayed Control Material Roche: 12000 cp/ml, log 4.01 Abbott: 23500 cp/ml log 4.37 Qiagen: 9570 cp/ml log: 3.98 Hypothetical Example created with Adobe Photoshop
Assayed Weight
NAT Calibration Paradox Instrument often trusted more than Control Different results for same sample on different systems considered normal
Is an assayed control acceptable?
Metrology The Science of Measurement Establishes measurement units Develops new measurement methods Provides measurement standards Traceability from standards to users Ensures suitability of measurement instruments Calibration and quality control of measurements
Cornerstones of Metrology Standardization Metrology Calibration Traceability (Value +/- uncertainty)
How to Calibrate a Scale Certified Calibrator used to check instrument Same calibrator can be used for different scales Calibrator has more credibility than instrument
Questions not asked How much does the 10 g calibration weight weigh on my scale? Do I get the same result on my new digital scale and my old balance?
Why does our industry sometimes ask? Which is correct value of the 1e5 IU/mL standard on my analyzer? Do I get with your material the same result on the system from company A versus the system from company B?
Inconsistent Results Non-Standardised Calibration Standard 1 System Company A PC1 Standard 2 System Company: B PC2 Standard 3 System Company: C PC3 Standard 4 System Company: D PC4 Current scenario for all analytes (e.g. CMV) except HIV-1, HBV and HCV
Limitations of Current NAT Control Materials (assayed, unassayed and kit controls) Not metrological based Often based on specific assay system Units not clearly defined Not independent (assayed & kit controls) Not useful for Standardisation
Consistent Results Standardised Calibration Instrument: A Traceable Calibrator or Standard Instrument: B Instrument: C Instrument: D Scenario for established measurement systems (e.g. scale)
Metrological Levels high low Level Traceable to SI Unit Int. reference measurement method See ISO 17511:2003 Introduction p. vii Calibrator Material Example 1 a) Yes Yes Yes Electrolytes 2 b) 1 No Yes Yes HbA 1c 3 b) 2 No Yes No 4 b) 3 No No Yes Haemostatic factors WHO Standard 5 b) 4 No No No MRSA
AcroMetrix Control Development AcroMetrix Quality System ISO 13485:2003 certified FDA 21CFR Part 820 Quality System Regulations (QSR) cgmp Calibrators and Controls values assigned following ISO 17511:2003 In vitro diagnostic medical devices Measurement of quantities in biological samples Metrological traceability of values assigned to calibrators and control materials
ISO17511: Traceability Traceability allows assay independent value assignment
Old School Measurement and Calibration Calibration of a Scale Certified Test Weights = Standards Traceable International Prototype Kilogram (IPK)
Example MRSA Development of a novel independent run control, compliant with ISO 17511
Hospital acquired drugresistant microorganisms Among patients with a secondary Staphylococcal infection diagnosis- those who had isolates tested for antimicrobial susceptibility had 52% lower probability of death before discharge, 17% lower mean length of stay (-2 days), and 22% lower cost than those who did not The majority (~50%) of infections is identified as methicillinresistant Staphylococcus aureus (MRSA) F. R. Lichtenberg: The value of Diagnostics: The Impact of Hospital Antimicrobial Susceptibility Testing. Prepared for The Advanced Medical Technology Association. March 2008
MRSA testing 1. Allows the physicians to administer appropriate and effective treatments 2. Reduces quarantine time/cost No international MRSA standard available, so how to: 1. Assess sensitivity? 2. Monitor day to day performance? 3. Control for extraction, amplification and detection
Metrological Level: 5 Requires In-house measurement procedure In-house calibrator Generate in-house unit according ISO 17511:2003 clause 4.1.1 d Level Traceable to SI Unit Int. reference measurement method Calibrator Material Example 1 a) Yes Yes Yes Electrolytes 2 b) 1 No Yes Yes HbA 1c 3 b) 2 No Yes No 4 b) 3 No No Yes Haemostatic factors WHO Standard 5 b) 4 No No No MRSA
Challenges for Development of a Control for MRSA Which strain should be used? Can we use whole bacteria? Which matrix to use? What is correct level for control?
Characteristics for a reliable MRSA control 1) quantified levels of bacterial cells 2) a low level of cells to interrogate the performance of the assay on a daily basis 3) close similarity to a patient sample 4) stable 5) high reproducibility (minimal vial-to-vial variability) 6) Preferably non infectious
MRSA strain for in-house working calibrator Hospital acquired Type: SCCmec Type 1 Strain: A900159 (Juuti) Non infectious Bacteria are inactivated in-house working calibrator unit: CFU/mL
MRSA strain for in-house working calibrator (2) From H.G. Schlegel, Allgemeine Mikrobiologie, 1985, 6. überarbeitete Auflage Strain: A900159 (Juuti): a phenotype demonstrating reduced aggregation (grape-shape formation) and increased consistency by Real-Time PCR compared to current Staphylococcus aureus strains used as external controls.
Quantification of an inactivated MRSA control From the viable MRSA stock the number of colonies grown on agar plates was determined (CFU/mL) The Ct value was determined for a low positive control on the Gene Xpert the amount of inactivated MRSA was determined to reach this Ct value. This amount was assigned to the value from the viable stock
Quantification through Ct values # bacteria CFU Ct correct shifted earlier
Cycle trheshold (Ct) Quantification through Ct values 40 35 30 25 20 5*E3 5*E4 15 10 5 0 MRSA MSSA
Mean Titer (CFU/ml) MRSA reproducibility 5000 4000 3000 2000 1000 0-1000 OptiQual ATCC 43300
Mean Ct value OptiQual MRSA Positive Control stability Accelerated Stability Tested with Cepheid GeneXpert 35,0 32,5 30,0 27,5 25,0 22,5 20,0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Time (weeks) OptiQual MRSA 45C OptiQual MRSA 37C OptiQual MRSA 25C According to the Arrhenius equation at least one year of stability at 2-8 C
Mean Ct Value OptiQual MRSA inter-laboratory performance Site Testing using Cepheid Gene Xpert 35,0 32,5 30,0 27,5 25,0 22,5 20,0 AcroMetrix (n = 8*) Scott and White Hospital (n = 8) Kingman Regional Medical Center (n = 4)
Conclusions OptiQual MRSA controls demonstrate superior precision Vs ATCC, enabling laboratories to monitor true assay variability. OptiQual MRSA control contains whole bacteria to monitor entire procedure to meet laboratory quality control requirements. Because of ISO17511 compliance, AcroMetrix is capable of rapidly developing different control levels, if required.
The Shape of Things to come?