Image processing mammography applications Isabelle Bloch Isabelle.Bloch@telecom-paristech.fr http://perso.telecom-paristech.fr/bloch LTCI, Télécom ParisTech Mammography p.1/27
Image processing for mammography 1. Answering needs for systematic screening, diagnosis, interventional applications. 2. Several imaging modalities. 3. Here: focus on X-ray mammography and tomosynthesis. Mammography p.2/27
A few words on imaging modalities Mammography: capability to image microcalcifications Mammography p.3/27
A few words on imaging modalities Echography: lesion differentiation, needle guidance Mammography p.3/27
A few words on imaging modalities MRI (using Gd): local extension assessment, diagnosis after treatment Mammography p.3/27
Typical views cranio-caudal medio-lateral-oblique Mammography p.4/27
Image processing chain From native image... Image correction: gain / offset defect pixels modulation transfer function compensation Post-processing: log transformation thickness equalization constrast enhancement CAD... Display: lighting monitor calibration VOI / LUT... to visualization Next: illustrations from S. Muller, GE Healthcare Mammography p.5/27
Image correction Mammography p.6/27
Image correction Mammography p.6/27
FTM compensation Mammography p.7/27
Thickness equalization Mammography p.8/27
Thickness equalization Mammography p.8/27
Thickness equalization Mammography p.8/27
Contrast enhancement Mammography p.9/27
Contrast enhancement Mammography p.9/27
Contrast enhancement Mammography p.9/27
Computer assisted detection: CAD Mammography p.10/27
Computer assisted detection: CAD Mammography p.10/27
CAD methods Filtering and enhancement: preferably using local methods local statistics, wavelets... compromise under-enhancement (can cause FN) / over-enhancement (FP) Segmentation: thresholding and region growing edge detection and deformable models template matching Markov random fields left/right differences multiscale fuzzy methods Quantitative measures: intensity, shape, texture, clusters Classification: artificial neural networks kernel-based methods (SVM...) decision trees Evaluation: specificity and sensitivity ROC curve: true positives as a function of false positives Mammography p.11/27
Tomosynthesis Mammography p.12/27
Tomosynthesis Mammography p.12/27
Tomosynthesis Mammography p.12/27
Tomosynthesis Greater conspicuity of lesions. Borders of lesions more clearly defined. Reduced call-back rate - almost eliminates recall for superimposed structures (summation shadows). Accurate 3-D location. Better differentiates benign from malignant. Mammography p.12/27
CAD for tomosynthesis PhD thesis of G. Peters, with GE Healthcare Back-Projection 3D Detection 3D Analysis 3D Decision A 2D Detection 2D Analysis Back-Projection Re-Projection 3D Analysis 3D Decision B 2D Detection 2D Analysis 2D Decision Back-Projection C Choice: Strategy B Mammography p.13/27
Algorithm scheme Raw Image Candidate Detection Fuzzy Segmentation Fuzzy Feature Extraction Partial Defuzzification Acquisition Geometry Back-Projection / Re-Projection Attribute Aggregation Classification Mammography p.14/27
Dense kernel detector Using wavelets and brackground density estimation: Mammography p.15/27
Segmentation result: circumscribed lesion Original image Reference contour Initialization Region-based Contour-based Hybrid Mammography p.16/27
Segmentation result: spiculated lesion Original image Reference contour Initialization Region-based Contour-based Hybrid Mammography p.17/27
Hypothesis testing for a radiological finding Detection Radiological Finding Hypothesis A: Circumscribed Mass Hypothesis B: Spiculated Mass Active Contour Model Circumscribed Mass Active Contour Model Spiculated Mass Fuzzy Active Contour Fuzzy Active Contour Feature Extraction Feature Extraction Fuzzy Attributes Fuzzy Attributes Fuzzy Decision Tree Fuzzy Decision Tree Confidence Degree in Hypothesis A Confidence Degree in Hypothesis B Decision Mammography p.18/27
Features from fuzzy contours Features: area, compacicty, mean gradient along the contour, homogeneity... Mammography p.19/27
Algorithm scheme for aggregation on particle level Projected Views Set of 2D Particles Set of 2D Fuzzy Attributes Partial Defuzzification Attribute Aggregation Fuzzy Particle Maps Correspondance Computation Cumulated Fuzzy Attribute Pixel Aggregation Re-Projected Fuzzy Particle Volume Fuzzy Particle Volume Re-Projection Mammography p.20/27
1 0.8 0.6 0.4 0.2 0 1 0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1 0.8 0.6 0.4 0.2 0 1 0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Complete processing chain original image wavelet filtering active contour segmentation assuming spiculated mass active contour segmentation assuming circumscribed mass wavelet filter response fuzzy active contour fuzzy active contour partial defuzzification feature extraction feature extraction partial defuzzification fuzzy particle map fuzzy attributes fuzzy attributes fuzzy particle map aggregation aggregation aggregation aggregation fuzzy particle volume cumulated fuzzy attributes cumulated fuzzy attributes fuzzy particle volume fuzzy decision tree fuzzy decision tree memberhip degree memberhip degree aggregation memberhip degree Mammography p.21/27
Automated detection of opacities and architectural distorsions in tomosynthesis PhD thesis of G. Palma, with GE Healthcare Mammography p.22/27
Global scheme: two channels approach DBT Volume Sub sampled volume Fuzzy 3D map Seeds Sub sampling Fuzzy connected filter (by slice) Thresholding A contrario detection (slice by slice processing) 2D and 3D aggregation Convergence regions Seeds Segmentation Contours Feature extraction Feature extraction Attributes Attributes Classification Classification Classes Masses detection Classes Architectural distortions detection Suspicious regions Mammography p.23/27
Makers from connected filters Mammography p.24/27
A contrario detection K c,q,r = (αr < cq < r) 1 if (tan(θ) cq αr) 0 otherwise. θ = angle between cq and orientation at point q. Z c,r = q Ω/αr< cq <r K c,q,r Mammography p.25/27
A contrario detection Z c,r λ r is ǫ-meaningful if the expectation of its number of occurrences in the image is less than ǫ (Desolneux et al., IJCV, 2000) λ r = min { λ N/P[Z c,r λ] ǫ } M where M it the number of pairs (c,r) to be considered. A contrario detection: computing {λr }, computing orientations, computing Zc,r for each (c,r), detection of ǫ-meaningful events (Zc,r > λ r ). Mammography p.25/27
A contrario detection Mammography p.25/27
Results Dense kernel detection: Mammography p.26/27
Results Convergence detection: Mammography p.26/27
Performances Performance of the whole dense kernel detection channel 1 0.8 Sensitivity 0.6 0.4 0.2 tag1 tag2 0 0 1 2 3 4 5 6 7 Number of false positives per breast Mammography p.27/27
Performances Performance of the a contrario detector for spiculated lesions only, and for architectural distortions and highly spiculated lesions 1 0.8 Sensitivity 0.6 0.4 0.2 0 0 20 40 60 80 100 120 Number of false positives per breast 1 0.8 Sensitivity 0.6 0.4 0.2 0 0 1 2 3 4 5 6 7 Number of false positives per breast Mammography p.27/27
Performances Performance of the suspicious convergence detection channel 1 0.8 Sensitivity 0.6 0.4 0.2 0 0 0.5 1 1.5 2 2.5 3 3.5 4 Number of false positives per breast Mammography p.27/27
Performances Performance of the complete detection process, after the aggregation step Sensitivity (%) Specificity (# of false positives per breast) 81.13 1.31 90.57 1.60 96.23 1.81 Plane of a DBT volume exhibiting a strongly spiculated lesion, used in the convergence channel evaluation. Although this lesion is not detected by this channel, it is correctly detected by the dense kernel channel, and therefore by the final fusion step. Mammography p.27/27