Holistically-Nested Edge Detection (HED)

Transcription

1 Holistically-Nested Edge Detection (HED) Saining Xie, Zhuowen Tu Presented by Yuxin Wu February 10, 20

2 What is an Edge? Local intensity change? Used in traditional methods: Canny, Sobel, etc Learn it! Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 2 / 15

3 What is an Edge? Local intensity change? Used in traditional methods: Canny, Sobel, etc Learn it! Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 2 / 15

4 Inspiration Fully Convolutional Network (FCN) Concept originally brought out for semantic segmentation No fully-connected layers (can be converted) Allow inputs of any sizes Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 / 15

5 HED Design Holistically-Nested architecture Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 4 / 15

6 HED Design Multiple Supervision Signals Single output, multiple cost Learn earlier, learn better Alleviate gradient vanishing Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 5 / 15

7 HED Design Convolutional Layers Fine-tuning from VGG: Lots of people do ine-tuning on top of VGG 5 stage x convolution only HED adds a side output (conv1x1) after each stage Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 6 / 15

8 HED Design Upsampling by Deconvolution Upsampling by a factor of k N + is implemented by a deconvolution with a 2k 2k kernel and output stride k An mathematically equivalent explanation (assume k = 2): 1 Input image with shape n 2 Zero-illed upsample as above, by a factor of 2 Shape becomes 2n 1 Convolve with a ilter Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 7 / with padding =, shape becomes (2n 1)+ = 2n+2 Then center-crop to 2n

9 HED Design Upsampling by Deconvolution Upsampling by a factor of k N + is implemented by a deconvolution with a 2k 2k kernel and output stride k An mathematically equivalent explanation (assume k = 2): 1 Input image with shape n 2 Zero-illed upsample as above, by a factor of 2 Shape becomes 2n 1 Convolve with a ilter Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 7 / with padding =, shape becomes (2n 1)+ = 2n+2 Then center-crop to 2n

10 HED Design Class-Balanced Sigmoid Cross Entropy Loss Sigmoid Cross Entropy Loss For each pixel, loss L = [y log(y)+(1 y ) log(1 y)] where ground truth label y 1 {0,1}, y = 1+e z In images, 90% pixels are not edge, cost function is dominated by negative labels To avoid this, re-weight the terms: Class-Balanced Sigmoid Cross Entropy Loss L = [βy log(y)+(1 β)(1 y ) log(1 y)] where β is the ratio of negative ground truth labels in this batch of data This loss function is computed for l 15 as well as l fuse = Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 8 / 15 5 α i l i i=1

11 HED Design Class-Balanced Sigmoid Cross Entropy Loss Sigmoid Cross Entropy Loss For each pixel, loss L = [y log(y)+(1 y ) log(1 y)] where ground truth label y 1 {0,1}, y = 1+e z In images, 90% pixels are not edge, cost function is dominated by negative labels To avoid this, re-weight the terms: Class-Balanced Sigmoid Cross Entropy Loss L = [βy log(y)+(1 β)(1 y ) log(1 y)] where β is the ratio of negative ground truth labels in this batch of data This loss function is computed for l 15 as well as l fuse = Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 8 / 15 5 α i l i i=1

12 HED Design Holistically-Nested architecture Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 9 / 15

13 Experiements Outputs Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, / 15

14 Experiements Qualitative Results Figure: Results on BSD500 (a small dataset) Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, / 15

15 Experiements Efect of Supervision Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, / 15

16 Experiements Efect of Supervision Figure: Output of 2nd stage with(left) and without(right) extra supervision Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 1 / 15

17 Experiements Misc Rotation/lip/scaling as data augmentation Using depth information (in NYUD dataset) gives better performance Pure FCN / HED without multiple supervision don t work as good 25 fps on K40 for input Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, / 15

18 Experiements CMU Pano Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, / 15

19 Thanks! Yuxin Wu Presented by Yuxin Wu Holistically-Nested Edge Detection (HED) February 10, 20 / 15