High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks

Transcription

1 High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks Krzysztof J. Geras Joint work with Kyunghyun Cho, Linda Moy, Gene Kim, Stacey Wolfson and Artie Shen. GTC 2017

2 WHERE DEEP LEARNING IS USEFUL amount of data available difficulty of the task

3 WHERE DEEP LEARNING IS USEFUL amount of data available natural image recognition speech machine recognition translation game playing the learning tasks for which deep learning makes a difference difficulty of the task

4 Can we save the world?

5 WHERE DEEP LEARNING IS USEFUL amount of data available natural image recognition speech machine recognition translation game playing medical image analysis? the learning tasks for which deep learning makes a difference difficulty of the task

6 BREAST CANCER SCREENING

7 BREAST CANCER SCREENING About 40 million exams performed yearly in the US.

8 BREAST CANCER SCREENING About 40 million exams performed yearly in the US. About 250 thousand women are diagnosed with cancer.

9 BREAST CANCER SCREENING About 40 million exams performed yearly in the US. About 250 thousand women are diagnosed with cancer. About 40 thousand die.

10 BREAST CANCER SCREENING R-MLO L-MLO R-CC L-CC (left cranial caudal) (right cranial caudal) (left mediolateral oblique) (right mediolateral oblique)

11 BREAST CANCER SCREENING We try to mimic predictions of a radiologist. Class 0: incomplete ( 15%). Class 1: negative ( 50%). Class 2: bening findings ( 35%).

12 BREAST CANCER SCREENING We try to mimic predictions of a radiologist. Class 0: incomplete ( 15%). Class 1: negative ( 50%). Class 2: bening findings ( 35%). Radiologists call these classes BI-RADS (short for Breast Imaging-Reporting and Data System).

13 CHALLENGES (1)

14 CHALLENGES (1) You need a lot of data to do deep learning.

15 CHALLENGES (1) You need a lot of data to do deep learning. Publicly available data sets contain about 1k images.

16 CHALLENGES (1) You need a lot of data to do deep learning. Publicly available data sets contain about 1k images. We build our own data set: 23k exams, 103k images. Each image is at least pixels.

17 CHALLENGES (2)

18 CHALLENGES (2)

19 CHALLENGES (2)

20 CHALLENGES (2)

21 CHALLENGES (2)

22 CHALLENGES (2)

23 CHALLENGES (2)

24 C HALLENGES (2) High resolution necessary - computational and engineering challenge.

25 CHALLENGES (3) Multi-view data. How to integrate information?

26 OUR MODEL Classifier p(y x) Concatenation (256 4 dim) DCN DCN DCN DCN L-CC R-CC L-MLO R-MLO

27 OUR MODEL Classifier p(y x) Concatenation (256 4 dim) DCN DCN DCN DCN L-CC R-CC L-MLO R-MLO layer kernel size stride #maps repetition global average pooling 256 convolution max pooling convolution max pooling convolution max pooling convolution convolution max pooling convolution input 1

28 RESULTS 1.0 true positive rate BI-RADS 0 BI-RADS 1 BI-RADS false positive rate AUC 0 vs. others: vs. others: vs. others: Average: 0.685

29 CONFIDENT TEST DATA We can compute the entropy of predictions, H(y x) = y C p(y x) log p(y x), and sort examples according to it.

30 CONFIDENT TEST DATA We can compute the entropy of predictions, H(y x) = y C p(y x) log p(y x), and sort examples according to it. We will consider the 30% with the lowest entropy to be confident.

31 RESULTS FOR CONFIDENT TEST DATA 1.0 true positive rate BI-RADS 0 BI-RADS 1 BI-RADS false positive rate AUC 0 vs. others: vs. others: vs. others: Average: 0.765

32 IMPACT OF DOWNSCALING AUC average AUC average AUC (confident) 1/8 1/4 1/2 1 resolution fraction

33 IMPACT OF THE DATA SET SIZE AUC average AUC average AUC (confident) 1/10 1/5 1/2 1 data set size fraction

34 VISUALISATION H(y x) We visualise x v, (i,j) where H(y x) = y C p(y x) log p(y x).

35 V ISUALISATION

36 V ISUALISATION

37 CONCLUSIONS We made a first step in the direction of end-to-end breast cancer screening with neural networks.

38 CONCLUSIONS We made a first step in the direction of end-to-end breast cancer screening with neural networks. It is much harder to learn the incomplete (0) class than other classes.

39 CONCLUSIONS We made a first step in the direction of end-to-end breast cancer screening with neural networks. It is much harder to learn the incomplete (0) class than other classes. We need to use the full resolution. A lot more effort is necessary to develop archiectures appropriate for data of large dimensionality.

40 CONCLUSIONS We made a first step in the direction of end-to-end breast cancer screening with neural networks. It is much harder to learn the incomplete (0) class than other classes. We need to use the full resolution. A lot more effort is necessary to develop archiectures appropriate for data of large dimensionality. We need more data.

41 CONCLUSIONS We made a first step in the direction of end-to-end breast cancer screening with neural networks. It is much harder to learn the incomplete (0) class than other classes. We need to use the full resolution. A lot more effort is necessary to develop archiectures appropriate for data of large dimensionality. We need more data. (We are currently processing a 10 times bigger data set).

42 Thank you! High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks K. J. Geras, S. Wolfson, S. G. Kim, L. Moy, K. Cho arxiv: