Do Your Flow Cytometric LDTs. Validation Guidelines? Fiona E. Craig, MD University of Pittsburgh School of Medicine

Transcription

1 Do Your Flow Cytometric LDTs Conform to the ICSH ICCS Validation Guidelines? Fiona E. Craig, MD University of Pittsburgh School of Medicine

2 How should LDTs be validated? Accuracy Specificity Sensitivity Precision Linearity Carryover Reportable range Reference range Stability

3 How should LDTs be validated?

4 How should Flow Cytometric LDTs be validated? Accuracy Specificity Sensitivity Precision Linearity Carryover Reportable range Reference range Stability?

5 Cell based Fluorescence Assays

6 Laboratory Designed Assay Validation Soluble analyte assay Determine single value xxxxxxxxxxxxxxxxxxxx Use stable reference material Limit of blank determined by omitting solute Precision determined by repeat testing Linearity performed by mixing solutions Cell based fluorescence assay Measure thousands of unique cellular parameters No stable reference material available Variable cell background / autofluorescence Repeat testing limited by short lived cell viability Linearity requires cells/beads of varying fluorescent intensity

7 Validation of cell base fluorescence assays. Cytometry Part B 84(5): 2013

8 Interpretation?

9 Chronic Lymphocytic Leukemia (CLL)

10 Features used to recognize CLL 1. B cell lymphoproliferative disorder (B LPD) 2. CLL versus other B LPD: a. Matutes score: CD5+, CD22 (dim +/ ), SmIg (dim +/ ), CD23+, FMC7 (score 4 or 5 = 87% CLL, 0.3% others) (score 0 or 1 = 0.4% CLL, 89% others) b. Improved score (Moreau): CD79b (SN8) instead of CD22 (score 4 or 5 = 89.3% CLL, 0.3% others) (score 0 or 1 = 0.0% CLL, 69.9% others) c. Lower levels of CD19 and CD20 in CLL (Ginaldi) Matutes E et al. Leukemia 8(10); : 1994 Moreau EJ et al. Am J Clin Pathol 108(4); : 1997 Ginaldi L et al. J Clin Pathol 51; : 1998

11 Features used to recognize CLL CD5+ B cells CD20 dim+ SmIg dim+/ CD23+

12 Assay Design & Optimization Goals of the flow cytometric assay: Identify all B cells, even those CD20 dim+/ : CD19 Determine if B cells CD5+: distinguish dim+ from background Determine if CD20 dim+/ : Other B cells reproducibly intermediate to bright Determine if SmIgdim+/ : Other B cells reproducibly intermediate to bright Determine if B cells CD23+: distinguish dim+ from background

13 Assay Design & Optimization Optimize Instrument Select Antibody / Fluorochrome combinations: Select bright fluorochromes for dim signals Avoid issues with data spreading Anticipate issues with tandem breakdown Optimize reagents: Confirm antibody specificity and avoid non specific phenomena Titrate to saturation / maximize signal to noise Design analysis strategy Establish interpretation and reporting guidelines

14 Distinguishing dim+/ versus + Antibody / Fluorochrome selection: Sufficient dynamic range between negative and normal B cell intensity (analytical sensitivity) Internal control B cells confirm expected staining of B cells Most internal control T cells indicate expected lack of staining Determine signal:noise ratio

15 Which is the brightest CD19 signal? A. B.

16 Which is the brightest CD19 signal? A. CD19 Median Fluorescence Intensity = 9,319 B. CD19 Median Fluorescence Intensity = 25,834

17 Which is the brightest CD19 signal? A. CD19 B MFI / T MFI = 176 B. CD19 B MFI / T MFI = 159

18 Which is the brightest CD19 signal? A. CD19 B MFI T MFI / 2 x rsd = 35 D W B. CD19 B MFI T / 2 x rsd = 26 D W

19 Determine if B cells CD5+ Antibody / Fluorochrome selection: CD19 PE Cy7 adequate intensity to identify all B cells? CD5 PerCP Cy5.5 adequate intensity to distinguish CD5 positive B cells from CD5 negative B cells (analytical sensitivity)? Any potential issues with data spreading?

20 Data Spreading Spreading of the negative due to measurement errors after compensation arising from multiple fluorescences spilling into each detector Dependent on instrument, configuration & performance, and reagents Can be predicted for any pair of detectors Should be considered during reagent selection to avoid limiting sensitivity Nguyen et at., Cytometry Part A 83A; : 2013

21 Data Spreading Spreading of the negative due to measurement errors after compensation arising from multiple fluorescences spilling into each detector Dependent on instrument, configuration & performance, and reagents Can be predicted for any pair of detectors Should be considered during reagent selection to avoid limiting sensitivity Nguyen et at., Cytometry Part A 83A; : 2013

22 Non specific phenomena Complement C1q mediates interactions between mouse IgG2 class antibodies Prevented by either complete serum removal or use of alternate IgG1 clones Potential source of artifact in whole blood lysis methods Wood & Levin. Cytometry Part B 70B; : 2006

23 40 µl Number Antibody Titering 20 µl 10 µl 5µL 0 50 Number Number Number CD3 FITC-A CD3 FITC-A CD3 FITC-A µl 1.2 µl 0.6 µl Number Number Number CD3 FITC-A CD3 FITC-A CD3 FITC-A 10 4 Signal:Noise Ratio ul of anti CD3 FITC Usesameconditions (volumes) as final assay Best signal to noise ratio Saturation desirable CD3 FITC-A 10 4 Courtesy Paul Wallace, Roswell Park Cancer Institute

24 Assay Performance Validation Qualitative Assay: Accuracy: comparison with expected results Specificity: analytical & clinical specificity = TN/(TN+FP) Sensitivity: analytical & clinical sensitivity = TP/(TP+FN) Precision: replicates of positive and negative Linearity Carryover Reportable range Reference range Stability: specimen and processed specimen Validation of cell base fluorescence assays. Cytometry Part B 84(5): 2013

25 Assay Performance Validation Clinical accuracy, specificity and sensitivity: Comparison of new assay versus another method (another lab., previous flow assay or morphology/ihc) 20 specimens no LPD, including PB, BM, LN, other 10 specimens CLL, including PB, BM, LN, other 10 specimens other LPD including PB, BM, LN, other Additional specimens if unexpected results, different anticoagulants Precision: 3 replicates 3 5 LPD & 3 5 no LPD (intra and inter assay) compare percent different populations (CV<10%) Stability (specimens, processed specimens, reagents): Time points (<1hr, 24 hrs., 48 hrs., or as determined by process) Must fall within precision Carryover: run no LPD, followed by LPD, then no LPD

26 CD5+ B cells Population with CLL like phenotype, but only small percent of total events. Clinical significance? CD20 dim+ SmIg dim+/

27 Monoclonal B Lymphocytosis Monoclonal B cell population persisting for at least 3 months total B cell count below 5 x 10 9 /L no other features diagnostic of a B cell lymphoid neoplasm CLL like and non CLL like MBL Prevalence CLL like MBL proportional to assay sensitivity Clinical significance related to absolute count: evaluation of lymphocytosis vs. population studies Higher count CLL like MBL: progression to CLL 1% per year decreased normal B cells leading to increased infection

28 CD5+ B cells MBL vs. CLL < 5 x 10 9 /L? CD20 dim+ SmIg dim+/

29 MBL vs. CLL? 20 peripheral blood specimens with CLL like B cells Results from single platform (SP) tube compared with 2 dual platform (DP) tubes: Kappa, lambda, CD5, CD10, CD19, CD38, CD45, CD20 FMC 7, CD23, CD19, CD5 3 discordant results relative to threshold of 5 x 10 9 /L Case Single Platform Dual Platform Tube A Tube B Rack No R Rack No R Clinical Setting MBL versus CLL Prior diagnosis CLL Subsequent diagnosis CLL Rollins Raval et al., Cytometry Part B 84B; : 2013

30 MBL vs. CLL? Rollins Raval et al., Cytometry Part B 84B; : 2013

31

32 Assay Performance Validation Quasi quantitative Assay: Accuracy: comparison with expected results Specificity: analytical & clinical specificity = TN/(TN+FP) Sensitivity: analytical & clinical sensitivity = TP/(TP+FN) Precision: replicates of positive and negative Linearity Carryover Reportable range Reference range Stability: specimen and processed specimen Validation of cell base fluorescence assays. Cytometry Part B 84(5): 2013

33 Minimal Residual CLL? CD5+ T cells CD5+ B cells

34 Minimal Residual CLL Evolving treatment of CLL: Prognostic markers identify higher risk patients Highly effective immunotherapy can eradicate MRD < 0.01%, or 0.001% MRD negative patients have a survival advantage Linear increase in progression free survival per log tumor depletion ( x 10 9 /L = 4 x 1.5 years) Improved detection of MRD: Molecular Flow cytometric

35 CLL MRD detection using CD19 / CD5 / kappa / lambda CD5+ B cells: Normal B cell subset (typically <30% B cells) Increases with regeneration, up to 90% Many CLL cells have only weak expression Light chain restriction: Highly skewed K:L ratio following therapy Accuracy of ratio limited by only a few events Confounded by admixed polytypic B cells

36 CLL MRD detection using CD19 / CD5 / kappa / lambda [CD5+ B cells > 82% & K:L ratio <0.005:1 or > 32:1] Positive predictive value 100%: 47% of cases (72% with >1% CLL, 37% with <1%) Negative predictive value poor: MRD+ in 42% with normal K:L ratio MRD+ in 31% with <10% CD5+ B cells Qualitative only AC Rawstron et al., Leukemia 27; : 2013

37 CLL MRD Assay Design & Optimization Informative antibodies: CD20 single marker with best separation Include CD5, CD20 and CD79b = 10x higher sensitivity cf. CD19 / CD5 / kappa / lambda CD5/CD19, CD20/CD38, CD81/22, CD79a/CD43 lowest inter laboratory variation and false + Fluorochrome antibody combinations Number events = 500,000 (0.01% = 50 events) Analysis strategy AC Rawstron et al., Leukemia 2; : 2007

38 CLL MRD Assay Design & Optimization CD19, CD20, CD22, CD81, CD79b, CD5, CD43, CD3

39 Platform independent reagent specification Preferred CD20 relative fluorescence intensity MFI B cells / MFI T cells >20 Rawstron AC et al on behalf of ERIC manuscript in preparation

40 CLL MRD Assay Performance Validation Quasi quantitative Assay: Accuracy Specificity Sensitivity: Analytical: limit of detection (LOD) / limit of blank (LOB) Functional: lower limit of quantitation (LLOQ) Precision Linearity Carryover Reportable range Reference range Stability Validation of cell base fluorescence assays. Cytometry Part B 84(5): 2013

41 CLL MRD Assay Performance Validation Dilutional studies: Verify desired target threshold (0.01% or 0.001%) Use WBC s from 5 specimens with CLL spiked at different levels into 5 normal specimens Assay over 24 hours, in 5 replicate and divide data into 5 files: Lower limit of blank: 95% negatives < target Lower limit of detection: 95% low positives > target Lower limit of quantitation: lowest level where replicates reliably meet criteria for precision Linearity across reportable range Stability

42 CLL dilutional study 1% CLL 0.1% CLL 0.01% CLL

43 How should Flow Cytometric LDTs be validated? More difficult than for many clinical laboratory tests Start by deciding on the purpose of the test: Qualitative, Quasi quantitative, Quantitative Develop & optimize assay to meet diagnostic needs Stop making changes! Follow ICSH ICCS guidelines for validation of cellbased fluorescence assays

44 Validation of cell base fluorescence assays. Cytometry Part B 84(5): 2013