Flow Cytometry Critical Assessment of Population Identification Methods

Transcription

1 Flow Cytometry Critical Assessment of Population Identification Methods Richard H. Scheuermann, Ph.D. Director of Informatics J. Craig Venter Institute

2 Single Cell Phenotypes Different cell types play different physiological roles in the body Cell identity and function (phenotype) is dictated by the subset of genes/proteins expressed Alterations in the normal expressed parts list can give rise to disease Bruce Wetzel & Harry Schaefer, National Cancer Institute

3 Flow Cytometry (FCM) a.k.a. Fluorescence Activated Cell Sorting (FACS TM ) Method: Stain cell population with fluorescent reagents that bind to specific molecules, e.g. fluorescein-conjugated anti-cd40 antibodies Measure fluorescence properties of each cell using flow cytometer Direct and indirect measurement of single cell characteristics, e.g. cell size, membrane protein expression, secreted protein expression, cell cycle state, DNA ploidy, signal transduction activation Research uses: study normal and abnormal cell activation, differentiation and function; biomarker discovery Clinical uses: diagnosis and monitoring of leukemia, lymphoma and myeloproliferative disorders

4 How a Flow Cytometer Works Fluidic System Optic System (s) Electronics

5 Flow Cytometry (FCM) a.k.a. Fluorescence Activated Cell Sorting (FACS TM ) Method: Stain cell population with fluorescent reagents that bind to specific molecules, e.g. fluorescein-conjugated anti-cd40 antibodies Measure fluorescence properties of each cell using flow cytometer Direct and indirect measurement of single cell characteristics, e.g. cell size, membrane protein expression, secreted protein expression, cell cycle state, DNA ploidy, signal transduction activation Research uses: study normal and abnormal cell activation, differentiation and function; biomarker discovery Clinical uses: diagnosis and monitoring of leukemia, lymphoma and myeloproliferative disorders

6 Traditional Flow Cytometry Analysis Goal - group together cells with similar characteristics Traditional approach - manual gating 2D at a time Subjective Time-consuming Doesn t handle overlapping distributions well Sensitive to slight difference in fluorescence intensity distributions between samples Requires at least one 2D plot that clearly segregates populations in question

7 FCM instrumentation & reagents FCM can measure many parameters simultaneously, e.g., BD LSR-II can produce data for up to 19 parameters for every cell in a given sample

8 CyTOF Mass Cytometry

9 Automated Cell Population Identification

10 FLOCK FLOCK is a density-based algorithmic method for the identification of unique cell populations in multi-dimensional FCM data

11 2D example

12 Divide with hyper-grids

13 Find dense hyper-regions

14 Merge neighboring dense hyper-regions

15 Clustering based on region centers

16 CD27 B220 CD38 17 B Cell Populations in Blood A PB GSM GNSM UM3-4 UM1-2 N1-3 DNM IgD CD24 IgG

17 Proportion Change of PB/Plasma Cells

18

19 FlowCAP Challenge Flow Cytometry: Critical Assessment of Population Identification Methods (FlowCAP) The goal of FlowCAP is to advance the development of computational methods for the identification of cell populations of interest in flow cytometry data by providing the means to objectively test these methods, initially by comparison to manual gating analysis by experts using common datasets. FlowCAP Challenges 1) Design specific computational challenges 2) Collect de-identified datasets and distribute to algorithm development community; 2) Collect challenge results from algorithm development community; 3) Assess results in comparison with some gold standard or defined performance metric

20 FlowCAP Challenges - Summary FlowCAP-I: Cell population identification i) completely automated, ii) manually tuned, iii) predefined population numbers, iv) trained using manual gates FlowCAP-II: Sample classification/fcm biomarkers i) HIV exposed but uninfected, ii) AML, iii) T cell responses following HIV vaccination FlowCAP-III: Collection of challenges i) rare cell population identification (EQAPOL), ii) survival biomarkers, iii) sample classification (HVTN ICS), iv) manual gating comparison (HIP-C lyoplate) FlowCAP-IV: Clinical outcome correlates Time to AIDS progression from PBMCs stimulated with HIV antigens

21 FlowCAP Challenges - Summary FlowCAP-I: Cell population identification i) completely automated, ii) manually tuned, iii) predefined population numbers, iv) trained using manual gates FlowCAP-II: Sample classification/fcm biomarkers i) HIV exposed but uninfected, ii) AML, iii) T cell responses following HIV vaccination FlowCAP-III: Collection of challenges i) rare cell population identification (EQAPOL), ii) survival biomarkers, iii) sample classification (HVTN ICS), iv) manual gating comparison (HIP-C lyoplate) FlowCAP-IV: Clinical outcome correlates Time to AIDS progression from PBMCs stimulated with HIV antigens

22 FlowCAP-I Datasets Graft versus Host Disease (GvHD) - samples for finding cellular signatures to predict or correlate with early detection of GvHD. Diffuse Large B-cell Lymphoma (DLBCL) lymph node biopsies from treated patients with histologically-confirmed DLBCL. Hematopoietic Stem Cell Transplant (HSCT) samples derived from mouse hematopoietic stem cell transplant experiments. Symptomatic West Nile Virus (WNV) PBMC from patients with symptomatic West Nile virus infection stimulated in vitro with WNV peptide pools. Normal Donors (ND) differences in the response of PBMC to various stimuli for a set of healthy donors, including both cell surface and intracellular markers. Dataset # Samples # Events # Colors # Markers, incl. FSC/SSC Provider GvHD 12 14, BCCRC & TreeStar DLBCL 30 5, BCCRC HSCT 30 10, BCCRC WNV , McMaster ND 30 17, Amgen

23 F-measure manual gating result algorithmic result precision = tp/(tp + fp) recall = tp/(tp + fn)

24 Comparison with Manual Gating

25 FlowCAP-I Results

26 Challenge 1 Completely Automated

27 FlowCAP-II Sample classification challenges, with sample labels provided for half of samples up front for training purposes Challenge 1: HIV exposed in utero and uninfected vs not exposed find cell populations that can discriminate between the two groups using blood samples taken 6 mo after birth and stimulated through TLRs Challenge 2: AML versus healthy blood and bone marrow with 6 different marker combinations Challenge 3: Identify antigen stimulation groups post HIV vaccine Gag and Env antigen stimulated T cells ex vivo using blood samples collected ~ 10 months after vaccination

28 FlowCAP-II Results Challenge 1 Too difficult? Can we really expect to predict in utero exposure to HIV without infection? Challenge 2 Too easy? Many methods showed perfect classification accuracy But...

29 Manual Gating Gold Standard

30 Misclassification With one exception, misclassification was randomly distributed Many methods misclassified the same sample (#340) as AML

31 FlowCAP Summary Several automated algorithms for population identification are able to closely match manual gating results with good performance Both model-based and non-model-based approaches performed well, but different methods produced better results on different datasets Merging results from multiple algorithms provided further improvements Excellent performance of methods to identify cell-based biomarkers for sample classification Manuscript published in Nature Methods

32 Project Applications Respiratory Pathogen Research Center (RPRC) T cell responses during severe RSV infection Biomarkers of poor respiratory function in premature infants La Jolla Institute for Allergy and Immunology Human Immune Profiling Center (LJI HIPC) T cell responses during latent and active Mtb infection and vaccination T cell responses during mild and severe Dengue virus infections UCSD Center for Advanced Laboratory Medicine (CALM) Diagnosis and prognosis of CLL and AML

33 Computational Analysis Pipeline Data transformation FCSTrans logicle Filtering DAFi Alignment GaussNorm Cell population identification FLOCK-cm Mapping FlowMap-FR Comparative analysis Wilcoxon RS original A_D117_filt.fcs A_D139_filt.fcs A_D140_filt.fcs A_D145_filt.fcs A_D160_filt.fcs A_D167_filt.fcs A_D73_filt.fcs A_D84_filt.fcs A_D8_filt.fcs A_U106_filt.fcs A_U150_filt.fcs A_U197_filt.fcs A_U29_filt.fcs A_U34_filt.fcs A_U36_filt.fcs A_U45_filt.fcs A_U50_filt.fcs A_U53_filt.fcs A_U68_filt.fcs A_U85_filt.fcs A_U95_filt.fcs N_D14_filt.fcs N_D30_filt.fcs N_D32_filt.fcs N_D34_filt.fcs N_D36_filt.fcs N_D49_filt.fcs N_D55_filt.fcs N_D86_filt.fcs N_D91_filt.fcs N_U104_filt.fcs N_U11_filt.fcs N_U14_filt.fcs N_U17_filt.fcs N_U187_filt.fcs N_U201_filt.fcs N_U59_filt.fcs N_U63_filt.fcs N_U7_filt.fcs N_U81_filt.fcs N_U9_filt.fcs S_D101_filt.fcs S_D106_filt.fcs S_D109_filt.fcs S_D111_filt.fcs S_D114_filt.fcs S_D126_filt.fcs S_D127_filt.fcs S_D130_filt.fcs S_D136_filt.fcs S_D142_filt.fcs S_D143_filt.fcs S_D146_filt.fcs S_D150_filt.fcs S_D155_filt.fcs S_D162_filt.fcs S_D31_filt.fcs S_D63_filt.fcs S_D72_filt.fcs S_D80_filt.fcs S_D87_filt.fcs S_U65_filt.fcs BV605_CCD6 normalized A_D117_filt.fcs A_D139_filt.fcs A_D140_filt.fcs A_D145_filt.fcs A_D160_filt.fcs A_D167_filt.fcs A_D73_filt.fcs A_D84_filt.fcs A_D8_filt.fcs A_U106_filt.fcs A_U150_filt.fcs A_U197_filt.fcs A_U29_filt.fcs A_U34_filt.fcs A_U36_filt.fcs A_U45_filt.fcs A_U50_filt.fcs A_U53_filt.fcs A_U68_filt.fcs A_U85_filt.fcs A_U95_filt.fcs N_D14_filt.fcs N_D30_filt.fcs N_D32_filt.fcs N_D34_filt.fcs N_D36_filt.fcs N_D49_filt.fcs N_D55_filt.fcs N_D86_filt.fcs N_D91_filt.fcs N_U104_filt.fcs N_U11_filt.fcs N_U14_filt.fcs N_U17_filt.fcs N_U187_filt.fcs N_U201_filt.fcs N_U59_filt.fcs N_U63_filt.fcs N_U7_filt.fcs N_U81_filt.fcs N_U9_filt.fcs S_D101_filt.fcs S_D106_filt.fcs S_D109_filt.fcs S_D111_filt.fcs S_D114_filt.fcs S_D126_filt.fcs S_D127_filt.fcs S_D130_filt.fcs S_D136_filt.fcs S_D142_filt.fcs S_D143_filt.fcs S_D146_filt.fcs S_D150_filt.fcs S_D155_filt.fcs S_D162_filt.fcs S_D31_filt.fcs S_D63_filt.fcs S_D72_filt.fcs S_D80_filt.fcs S_D87_filt.fcs S_U65_filt.fcs BV605_CCD6 original A_D117_filt.fcs A_D139_filt.fcs A_D140_filt.fcs A_D145_filt.fcs A_D160_filt.fcs A_D167_filt.fcs A_D73_filt.fcs A_D84_filt.fcs A_D8_filt.fcs A_U106_filt.fcs A_U150_filt.fcs A_U197_filt.fcs A_U29_filt.fcs A_U34_filt.fcs A_U36_filt.fcs A_U45_filt.fcs A_U50_filt.fcs A_U53_filt.fcs A_U68_filt.fcs A_U85_filt.fcs A_U95_filt.fcs N_D14_filt.fcs N_D30_filt.fcs N_D32_filt.fcs N_D34_filt.fcs N_D36_filt.fcs N_D49_filt.fcs N_D55_filt.fcs N_D86_filt.fcs N_D91_filt.fcs N_U104_filt.fcs N_U11_filt.fcs N_U14_filt.fcs N_U17_filt.fcs N_U187_filt.fcs N_U201_filt.fcs N_U59_filt.fcs N_U63_filt.fcs N_U7_filt.fcs N_U81_filt.fcs N_U9_filt.fcs S_D101_filt.fcs S_D106_filt.fcs S_D109_filt.fcs S_D111_filt.fcs S_D114_filt.fcs S_D126_filt.fcs S_D127_filt.fcs S_D130_filt.fcs S_D136_filt.fcs S_D142_filt.fcs S_D143_filt.fcs S_D146_filt.fcs S_D150_filt.fcs S_D155_filt.fcs S_D162_filt.fcs S_D31_filt.fcs S_D63_filt.fcs S_D72_filt.fcs S_D80_filt.fcs S_D87_filt.fcs S_U65_filt.fcs BV605_CCD6 normalized A_D117_filt.fcs A_D139_filt.fcs A_D140_filt.fcs A_D145_filt.fcs A_D160_filt.fcs A_D167_filt.fcs A_D73_filt.fcs A_D84_filt.fcs A_D8_filt.fcs A_U106_filt.fcs A_U150_filt.fcs A_U197_filt.fcs A_U29_filt.fcs A_U34_filt.fcs A_U36_filt.fcs A_U45_filt.fcs A_U50_filt.fcs A_U53_filt.fcs A_U68_filt.fcs A_U85_filt.fcs A_U95_filt.fcs N_D14_filt.fcs N_D30_filt.fcs N_D32_filt.fcs N_D34_filt.fcs N_D36_filt.fcs N_D49_filt.fcs N_D55_filt.fcs N_D86_filt.fcs N_D91_filt.fcs N_U104_filt.fcs N_U11_filt.fcs N_U14_filt.fcs N_U17_filt.fcs N_U187_filt.fcs N_U201_filt.fcs N_U59_filt.fcs N_U63_filt.fcs N_U7_filt.fcs N_U81_filt.fcs N_U9_filt.fcs S_D101_filt.fcs S_D106_filt.fcs S_D109_filt.fcs S_D111_filt.fcs S_D114_filt.fcs S_D126_filt.fcs S_D127_filt.fcs S_D130_filt.fcs S_D136_filt.fcs S_D142_filt.fcs S_D143_filt.fcs S_D146_filt.fcs S_D150_filt.fcs S_D155_filt.fcs S_D162_filt.fcs S_D31_filt.fcs S_D63_filt.fcs S_D72_filt.fcs S_D80_filt.fcs S_D87_filt.fcs S_U65_filt.fcs BV605_CCD6

34 Diagnostic Application: Chronic Lymphocytic Leukemia (CLL)

35 Leukemia staining panels 2016

36 10 color 4 color CLL challenges/opportunities Difficult to cleanly separate CLL due to overlapping expression, especially in MRD Prognostics significance of different T cell subsets Significance of CLL subpopulations Increased complexity of results using 10 color panel

37 CLL study design CLL (11), no CLL (5) and MRD (4) samples from CALM Two 10-color CLL staining panels Initial filter on singlets, viable cells, and lymphocytes using DAFi Then filter on CLL (CD5+CD19+), T cell (CD5+CD19-), and B cell (CD5- CD19+) subsets using DAFi Cell population identification using FLOCK Panel #1 and CLL analysis only Results: Improved marker-based definition of CLL Better monitoring of MRD CLL subtype identification

38 CALM SOP

39 CD19 DAFi filtering results B cells T cells CLL cells Sample #58 DAFi filters Composite FLOCK results CD5

40 CLL results Identified 45 distinct cell populations using Panel #1 and initial CLL filter 10 of these are probably false positives due to presence in normal samples => refine CLL definition to CD5 + CD19 + CD45 + CD10 - CD79b int/- Of the remaining are significantly different between CLL and normal 7 appear to be specific to one CLL case; these may represent distinct CLL subtypes

41 CLL subset examples Normal CLL 5br 5di 5di 5di/br 5di/br

42 Improved CLL definition: CD10-CD79int/- Normal CLL

43 DAFi Filtering: Original vs. New 66-normal 54-CLL 13-MRD

44 CD79b CD19 CD10 CLL samples with improved CLL definition CD5

45 CD79b CD19 CD10 Normal samples with improved CLL definition CD5

46 Minimal residual disease 66-normal 74-CLL 13-MRD 23-MRD 44-MRD

47 MRD samples with improved CLL definition X: CD5; Y: CD19 X: CD10: Y: CD79b

48 Summary New computational and statistical methods (e.g. FLOCK, SWIFT, FLAME, flowmeans, OpenCyto) are becoming part of routine FCM data analysis and are replacing manual gating, especially for high dimensional data But caveat emptor...some methods are better than others Don t rely on summary statistics alone...look at the results For cell population identification methods Each population show a unimodal distribution for all evaluated marker Marker expression patterns should show natural distibutions Beware of over-partitioning FlowCAP challenges are providing objective means to judge the quality of the analysis results Application for improved diagnosis of CLL Improved definition CD19+CD5+ => CD19+CD5+CD10-CD79bdim Improved diagnostic accuracy Improved monitoring of MRD Subtype classification Prognostic significance?

49 Acknowledgments J. Craig Venter Institute Yu Max Qian Alex Lee Hyunsoo Kim Rick Stanton Joyce Hsiao Katie O Nell University of California, San Diego Jack Bui Broad Institute of MIT and Harvard Jill Mesirov Southern Methodist University Monnie McGee Mengya Liu La Jolla Institute of Allergy & Immunology Alex Sette Bjoern Peters Cecilia Arlehamn University of Rochester Ignacio Sanz David Topham Texas Advanced Computing Center Weijia Xu San Diego Supercomputer Center Robert Sinkovits Ilkay Altintas Jianwu Wang Shweta Purawat FlowCAP Organizing Committee Nima Aghaeepour, Stanford Ryan Brinkman, BCCA Greg Finak, FHCRC Raphael Gottardo, FHCRC Tim Mosmann, URMC Richard H. Scheuermann, JCVI Supported by NIH N01AI40076, R01EB008400, HHSN C, U19AI118626