Advanced Artificial Intelligence Lecture 5: Neural networks
Advanced Artificial Intelligence Lecture 5: Neural Networks
Outline Perceptron Introduction Deep Neural Network Structure Backpropagation
2 Outline ▪ Perceptron Introduction ▪ Deep Neural Network Structure ▪ Backpropagation
Perceptron Introduction impulses carried toward cell body branches dendrites of axon axon axon nucleus terminals impulses carried away from cell body domestication domestication axon from a neuron se synapse cell body w1 ∑mx+b output axo activation u22 domestication Perceptron is Inspired by Neuron a diagram showing a perceptron updating It is a classifier its linear boundary as more training examples are added
3 Perceptron Introduction Perceptron is Inspired by Neuron. It is a classifier. A diagram showing a perceptron updating its linear boundary as more training examples are added
Single layer one-input Perceptron Input 0, Linear Combiner Limiter Output ∑ Y Threshold 0,0 OR a=Wx, b 8(r) 1,x≥0 1.5,b=-0.5 Y=6(a) 0,x<0 1
4 Single layer one-input Perceptron 𝑎 = 𝑤1𝑥1 + 𝑏 𝑌 = 𝜃(𝑎ሻ 𝜃(𝑥ሻ = ቊ 1, 𝑥 ≥ 0 0, 𝑥 < 0 𝑤1 = −1.5, 𝑏 = −0.5 𝑏
Single layer multi-input Perceptron Inputs L Hard Combiner Limiter Output Positive region 2·」 Threshold 1,w2=1b W1X1+W2x2+ b +a·E·x Y=(a) b=b+a·E Single layer perceptron is a linear E=Y-Y classifier A(x 0,x<0
5 Single layer multi-input Perceptron 𝑎 = 𝑤1𝑥1 + 𝑤2𝑥2 + 𝑏 𝑌 = 𝜃(𝑎ሻ 𝑤𝑖 = 𝑤𝑖 + 𝛼 ⋅ 𝐸 ⋅ 𝑥𝑖 𝑏 = 𝑏 + 𝛼 ⋅ 𝐸 𝐸 = 𝑌 − 𝑌 𝜃(𝑥ሻ = ቊ 1, 𝑥 ≥ 0 0, 𝑥 < 0 Single layer perceptron is a linear classifier 𝑏
Single Hidden layer multi-input Perceptron b A Multiple inputs, single hidden node perceptron Still a linear classifier, with a hyper-classify plane
6 Single Hidden layer multi-input Perceptron Multiple inputs, single hidden node perceptron. Still a linear classifier, with a hyper-classify plane
Non-linear activation Perceptron Inputs Linear sigmoid Combiner 2) Threshold o( is a non-linear activation function, sigmoid was the most popular one aEWx,tw,xat b o(y)= 1+e o(a
7 Non-linear activation Perceptron 𝑎 = 𝑤1𝑥1 + 𝑤2𝑥2 + 𝑏 𝑏 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 𝑌 = 𝜎(𝑎ሻ
Non-linear activation Perceptron YA b A b With sigmoid activation function 8
8 Non-linear activation Perceptron With sigmoid activation function
Outline Perceptron Introduction Deep Neural Network Structure Backpropagation
9 Outline ▪ Perceptron Introduction ▪ Deep Neural Network Structure ▪ Backpropagation
Deep Neural Network One neuron(perceptron): Linear separation Multi-hidden layers non-linear One hidden layer: Realization of convex regions activation All the complex shapes Two hidden layers: Realization of non-convex regions 10
10 Deep Neural Network One neuron (perceptron): Linear separation One hidden layer: Realization of convex regions Two hidden layers: Realization of non-convex regions Multi-hidden layers non-linear activation: All the complex shapes