Fast R-CNN. Ross Girshick Facebook AI Research (FAIR) Work done at Microsoft Research. Presented by: Nick Joodi Doug Sherman

Transcription

1 Fast R-CNN Ross Girshick Facebook AI Research (FAIR) Work done at Microsoft Research Presented by: Nick Joodi Doug Sherman

2 Fast Region-based ConvNets (R-CNNs) Fast Sorry about the black BG, Girshick s slides were all black. 2

3 The Pascal Visual Object Classes Challenge Overview Classification, Detection, Segmentation For each image: Does it contain the class? classification Where is it? detection via bounding box Evaluation Mean Average Precision (map) Participants submitted results in the form of confidence Produce Precision Recall curves Average precision for each class Take mean to get map 3

4 Object detection renaissance (2013-Present) 4

5 Object detection renaissance (2013-Present) 5

6 Object detection renaissance (2013-Present) 6

7 Agenda 1. Pre-existing Models a. b. 2. Ways to improve a. b. 3. Slow R-CNN SPP-net SGD Mini-Batch New Loss Function Fast R-CNN a. b. Architecture Results & Future Work 7

8 Region-based convnets (R-CNNs) R-CNN (aka slow R-CNN ) [Girshick et al. CVPR14] SPP-net [He et al. ECCV14] 8

9 9

10 10

11 11

12 12

13 13

14 14

15 What s wrong with slow R-CNN? Ad hoc training objectives Fine-tune network with softmax classifier (log loss) Train post-hoc linear SVMs (hinge loss) Train post-hoc bounding-box regressors (L2 loss) 15

16 What s wrong with slow R-CNN? Ad hoc training objectives Fine-tune network with softmax classifier (log loss) Train post-hoc linear SVMs (hinge loss) Train post-hoc bounding-box regressors (L2 loss) Training is slow (84h), takes a lot of disk space 16

17 What s wrong with slow R-CNN? Ad hoc training objectives Fine-tune network with softmax classifier (log loss) Train post-hoc linear SVMs (hinge loss) Train post-hoc bounding-box regressors (L2 loss) Training is slow (84h), takes a lot of disk space Inference (detection) is slow 47s / image with VGG16 [Simonyan & Zisserman. ICLR15] Fixed by SPP-net [He et al. ECCV14] 17

18 Agenda 1. Pre-existing Models a. b. 2. Ways to improve a. b. 3. Slow R-CNN SPP-net SGD Mini-Batch New Loss Function Fast R-CNN a. b. Architecture Results & Future Work 18

19 19

20 20

21 21

22 22

23 23

24 24

25 Pyramid Pooling Layer (w/4 x h/4) (2 x 1) To FC (w/2 x h/2) (4 x 2) (w/1 x h/1) (8 x 2) Region Stride/Window Size Output of Pooling Concatenated 25

26 26

27 What s wrong with SPP-net? Inherits the rest of R-CNN s problems Ad hoc training objective Training is slow (25h), takes a lot of disk space Introduces a new problem: cannot update parameters below SPP layer during training 27

28 28

29 Agenda 1. Pre-existing Models a. b. 2. Ways to improve a. b. 3. Slow R-CNN SPP-net SGD Mini-Batch New Loss Function Fast R-CNN a. b. Architecture Results & Future Work 29

30 SGD Mini-Batch Method for RoIs 30

31 SGD Mini-Batch Method for RoIs 31

32 SGD Mini-Batch Method for RoIs Input size for SPP-net 32

33 SGD Mini-Batch Method for RoIs 33

34 SGD Mini-Batch Method for RoIs 34

35 SGD Mini-Batch Method for RoIs 35

36 SGD Mini-Batch Method for RoIs 36

37 SGD Mini-Batch Method for RoIs 37

38 Agenda 1. Pre-existing Models a. b. 2. Ways to improve a. b. 3. Slow R-CNN SPP-net SGD Mini-Batch New Loss Function Fast R-CNN a. b. c. Architecture Results Future Work 38

39 Revised loss function For the classification For the bounding box p: Predicted RoI Classification u: True RoI Classification tu = (tx,ty,tw,th): Predicted Bounding Box v = (vx,vy,vw,vh): True Bounding Box ƛ : Controls the balance between the two losses 39

40 Revised loss function 40

41 Revised loss function Smooth: Continuously Differentiable 41

42 Agenda 1. Pre-existing Models a. b. 2. Ways to improve a. b. 3. Slow R-CNN SPP-net SGD Mini-Batch New Loss Function Fast R-CNN a. b. Architecture Results & Future Work 42

43 Fast R-CNN Fast test-time, like SPP-net One network, trained in one stage Higher mean average precision than slow R-CNN and SPP-net 43

44 44

45 45

46 46

47 47

48 48

49 49

50 50

51 Agenda 1. Pre-existing Models a. b. 2. Ways to improve a. b. 3. Slow R-CNN SPP-net SGD Mini-Batch New Loss Function Fast R-CNN a. b. Architecture Results & Future Work 51

52 52

53 53

54 54

55 What s still wrong? Out-of-network region proposals Selective search: 2s / img; EdgeBoxes: 0.2s / img Fortunately, this has already been solved S. Ren, K. He, R. Girshick & J. Sun. Faster R'CNN: Towards Real'Time Object Detection with Region Proposal Networks. NIPS (2015). 55

56 Fast R-CNN take-aways End-to-end training of deep ConvNets for object detection Fast training times Open source for easy experimentation A large number of ImageNet detection and COCO detection methods are built on Fast R-CNN 56