Visual interpretation in pathology

Transcription

1 13

2 Visual interpretation in pathology Tissue architecture (alteration) evaluation e.g., for grading prostate cancer Immunohistochemistry (IHC) staining scoring e.g., HER2 in breast cancer (companion diagnostic marker for Herceptin therapy)

3 Digital shift Image acquisition Image processing & analysis Measure extraction Statistics Data Mining

4 Batch of 210 slides Whole Slide Imaging (WSI) Multiresolution tiled image file

5 Digital shift Image acquisition Image Processing processing & Analyse analysis Measure extraction Statistics Data Mining

6 Digital Image Processing Aims To provide computer-aided tools for quantitative analysis in order to extract objective numerical data on tissue structures and staining patterns Advantages Standardized morphologic and staining measurement Automation Improved accuracy and efficiency Requirements Multidisciplinary and integrated approach

7 Premise of standardized measurements: Image normalization Why: variability in color and intensity with different sources (staining protocol, reagents, scanning device, ) Batch 1 Batch 2 Consequences: biased measurements strong limitations in digital image processing automation

8 Image normalization pipeline for IHC staining images Van Eycke YR et al., submitted for publication

9 Batch 1 = Ref Batch 2 Batch 2 Norm Batch 2 Quantitative characterization (e.g. labeling index) Van Eycke YR et al., submitted for publication Batch 1

10 Even weak variations (which do not affect pathologists) may impact measures automatically extracted by means of image processing Batch 1 Batch 2 Batch 2 Norm Quantitative characterization (e.g. labeling index) Van Eycke YR et al., submitted for publication

11 Morphometric and textural analysis Characterization of tissue architecture, histological structure, cell density, etc. Bhargava R, Madabhushi A. Annu. Rev. Biomed. Eng

12 Identification of regions of interest for subsequent morphological or staining analyses Tumor cell identification in skin tissue using textural features and supervised classification

13 Staining analysis Immunohistochemistry: positive vs. negative nuclei, cytoplasm, lymphocytes, etc.

14 CISH, CR NA ISH : require merging z-stack acquisition UCA-1 RNA in bladder cancer (9 slides merged)

15 Very promising approach: Deep learning component able to learn from data Representation learning (e.g. artificial neural networks, random forests) : important features can be automatically discovery from raw data (i.e. images) Deep learning = representation learning with multiple levels of features Each level = abstract features discovered from the previous level Need a lot of data theoretical-motivations-deep-learning.html

16 Deep Learning = Learning Hierarchical Representations Deep learning: several stages of non-linear feature transformation It's deep if it has more than one stage of non-linear feature transformation Low-level feature Low-Level Feat ure Mid-level feature Mid-Level Feat ure High-level feature High-Level Feat ure Trainable Classifier Classifier eature visualization of convolutional net trained on I magenet from [ Zeiler & Fergus 2013] Zeiler & Fergus ECCV 2014

17 Fig. 12: CNN architecturesof thecolon gland segmentat ion approach in [22]. (a) Shows the architecture of Object-Net able while to identify (b) complex shows the histological architecture of Seperat or Net. Bot h are modelled structures after the from LeNet-5 a set of training CNN. images able to extract new (subvisual) The output of each CNN was individually histological fed to separat features e softmax functions to produce two probability distributions. In the third step, the output probability distributions were combined to assign the final class labels to a pixel. To train the CNNs, images from the dat aset were K. rot Sirinukunwattana, ed. Furthermore, et al. 2016, Gland to improve segmentation the in colon histology images: The GlaS challenge contest, execution speed of the proposed method, the original 775 by 522 pixels images Med Image Anal. Jan 2017

18 Bhargava R, Madabhushi A. Annu. Rev. Biomed. Eng

19 Digital shift Image acquisition Image processing & analysis Measure extraction Statistics Data Mining

20 Large potential for quantitative histopathological features Biomarker = 1 biological target, e.g. an antigen revealed by IHC an molecular target revealed by CISH, CRISH Extraction of features: standard number, area, intensity architecture and spatial topology cluster, distribution, cell organization, object shape analysis To evaluate (each) at diagnosis, prognosis and therapy-response levels

21 Conventional staining characterization LI = 20.2%

22 Drawbacks of conventional staining characterization High tissue heterogeneity No distinction between staining patterns LI = 20.2% LI = 20.5%

23 Novel features for biomarker analysis Characterization of spatial heterogeneity of nuclear staining patterns e.g., detection of highly-proliferative regions, i.e. Ki67 Hot- Spots, in high-grade gliomas 0.70x 10x B Moles Lopez X. et al., Cytometry 2012

24 HOT-SPOT detection by a clustering method Result visualization Possible manual adjustment by pathologists Extraction of quantitative features (number, size, density, etc.) Moles Lopez X. et al., Cytometry

25 Tissue and cellular organization Graph-based measurements: gland proximity, shape and nuclear density for determining the grade of colorectal dysplasia. Peter W. Hamilton, et al. Methods, Volume 70, Issue 1, 2014, 59 73

26 Multiple biomarkers: to combine information Image registration Colocalisation / co-expression analysis Ki-67 (DAB) Galectin-3 (Fast red) Virtual double staining Real double staining Moles Lopez X. et al., J Am Med Inform Assoc. 2015

27 Biomarker colocalisation: Registration & feature extraction workflow Colocalisation map: local contribution Mo i = QS i LI i 2 2 åqs i LI i å Colocalisation index: whole ROIs 0 Ro= å( QS i LI i ) 2 2 åqs i LI i å 1

28 Candidate biomarker validation: high throughput analysis using tissue microarray (TMA) many hundreds of patient samples in a single assay evaluation of clinical utility Van Eycke YR, et al. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2015

29 Biomarker colocalization: Registration accuracy 5 µm Van Eycke YR, et al. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2015

30 Digital shift Image acquisition Image processing & analysis Measure extraction Statistics Data Mining

31 Detection Selection Registration Tissue Staining Regions of interest (ROI) Validation by pathologists Maps Measures Number, area & intensity features Topology & architecture Colocalization Biostatistics Data mining Feature selection Knowledge extraction Decision aid Biomarker validation

32 Clinical database Treatments Demographic data Past history Symptoms Clinical DB Exams Diagnosis Follow-up

33 Data integration and analysis Diagnosis Treatment Follow-up Feature selection Clustering Classification Survival analysis Biomarker evaluation Risk assessment Decision aid Patient stratification Clinical DB Histomorpho DB Molecular DB 33

34 Prediction of recurrence in cancer patients Feulgen, H&E staining Nuclear features shape architecture orientation texture Bhargava R, Madabhushi A. Annu. Rev. Biomed. Eng

35 Companion biomarker discovery and validation for stratified medicine Peter W. Hamilton, et al. Methods, Volume 70, Issue 1, 2014, 59 73

36 Li et al., Briefings in Bioinformatics, 2016, 1 13 INTEGROMICS framework

37 Thank you DIAPath team (present and former members) Collaborators