Advanced Artificial Intelligence Lecture 6: Convolutional Neural network
Advanced Artificial Intelligence Lecture 6: Convolutional Neural Network
Outline Convolutional neural Network Convolution Max Pooling CNN Forward Propagation CNN Backward Propagation CNN Architectures ■ LeNet-5、 AlexNet VGGNet GooqLeNet ■ ResNet
2 Outline ▪ Convolutional Neural Network ▪ Convolution ▪ Max Pooling ▪ CNN Forward Propagation ▪ CNN Backward Propagation ▪ CNN Architectures ▪ LeNet-5 、 AlexNet ▪ VGGNet ▪ GoogLeNet ▪ ResNet
[Zeiler, M. D, ECCV 2014 Why cnn for Image? Represente d as pixels The most basic Use 1st layer as module to Use 2nd layer as classifiers build classifiers module Can the network be simplified by considering the properties of images? Sourceoftheslidehttp://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/cnn.pdf
3 Why CNN for Image? [Zeiler, M. D., ECCV 2014] Can the network be simplified by considering the properties of images? 1 x 2 x … … Nx … … … … … … … … … … … … The most basic classifiers Use 1st layer as module to build classifiers Use 2nd layer as module …… Represente d as pixels Source of the slide: http://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/CNN.pdf
Why cnn for Image? Some patterns are much smaller than the whole Image Aneuron does not have to see the whole image to discover the pattern Connecting to small region with less parameters “ beak, detector Sourceoftheslidehttp://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/cnn.pdf
4 Why CNN for Image? ▪ Some patterns are much smaller than the whole image A neuron does not have to see the whole image to discover the pattern. “beak” detector Connecting to small region with less parameters Source of the slide: http://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/CNN.pdf
Why cnn for Image? The same patterns appear in different regions upper-left beak” detector Do almost the same thing They can use the same set of parameters middle beak” detector 5 Sourceoftheslidehttp://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/cnn.pdf
5 Why CNN for Image? ▪ The same patterns appear in different regions. “upper-left beak” detector “middle beak” detector They can use the same set of parameters. Do almost the same thing Source of the slide: http://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/CNN.pdf
Why cnn for Image? Subsampling the pixels will not change the object bird bird subsampling We can subsample the pixels to make image smaller Less parameters for the network to process the image Sourceoftheslidehttp://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/cnn.pdf
6 Why CNN for Image? ▪ Subsampling the pixels will not change the object subsampling bird bird We can subsample the pixels to make image smaller Less parameters for the network to process the image Source of the slide: http://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/CNN.pdf
The whole cnn cat do Convolution Max Pooling Can repeat Fully Connected many times Feedforward network Convolution OOOOOOOU Max Pooling Flatten Sourceoftheslidehttp://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/cnn.pdf
7 The whole CNN Fully Connected Feedforward network cat dog …… Convolution Max Pooling Convolution Max Pooling Flatten Can repeat many times Source of the slide: http://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/CNN.pdf
The whole cnn Property 1 Some patterns are much smaller Convolution than the whole image Property 2 The same patterns appear in Max Pooling different regions Can repeat many times Property 3 Convolution Subsampling the pixels will not change the object Max Pooling Flatten 8 Sourceoftheslidehttp://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/cnn.pdf
8 The whole CNN Convolution Max Pooling Convolution Max Pooling Flatten Can repeat many times ➢ Some patterns are much smaller than the whole image ➢ The same patterns appear in different regions. ➢ Subsampling the pixels will not change the object Property 1 Property 2 Property 3 Source of the slide: http://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/CNN.pdf
The whole cnn cat d Convolution Max Pooling Can repea Fully Connected many times Feedforward network Convolution Max Pooling Flatten 9 Sourceoftheslidehttp://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/cnn.pdf
9 The whole CNN Fully Connected Feedforward network cat dog …… Convolution Max Pooling Convolution Max Pooling Flatten Can repeat many times Source of the slide: http://219.216.82.193/cache/8/03/speech.ee.ntu.edu.tw/43149163c97eb6be7590e3d8de445a67/CNN.pdf
Outline Convolutional neural Network Convolution Max Pooling CNN Forward Propagation CNN Backward Propagation CNN Architectures ■ LeNet-5、 AlexNet VGGNet GooqLeNet ■ ResNet 10
10 Outline ▪ Convolutional Neural Network ▪ Convolution ▪ Max Pooling ▪ CNN Forward Propagation ▪ CNN Backward Propagation ▪ CNN Architectures ▪ LeNet-5 、 AlexNet ▪ VGGNet ▪ GoogLeNet ▪ ResNet
