梯度下降的原理 Minimize the cost function C(x1,x2),and following Calculus,C changes as follows: Ax+0x2 △C≈0x x2 Define△r=(r,△x2)',VC=(S,)',and get:: △C≈VC·△x To make△C negative,set△xr=-nVC,and get: △C≈-nVC.7C=-ll7C2≤0 Then fromx+1-x,=Ax =-nVC,get: X1+1=xI-nVC Gradient is fastest descent direction only,as △C≈7C·△x=C△cos(0)≤=0，π/2≤9≤π3/2 8/51梯度下降的原理 ▶ Minimize the cost function C(x1, x2)，and following Calculus, C changes as follows: ∆C ≈ ∂C ∂x1 ∆x1 + ∂C ∂x2 ∆x2 ▶ Define ∆x = (∆x1, ∆x2) T , ∇C = ( ∂C ∂x1 , ∂C ∂x2 ) T , and get: ∆C ≈ ∇C · ∆x ▶To make ∆C negative, set ∆x = −η∇C, and get: ∆C ≈ −η∇C · ∇C = −η∥∇C∥ 2 ≤ 0 ▶Then from xt+1 − xt = ∆x = −η∇C, get: xt+1 = xt − η∇C ▶ Gradient is fastest descent direction only, as ∆C ≈ ∇C · ∆x = ∥∇C∥∥∆x∥ cos(θ) ≤= 0, π/2 ≤ θ ≤ π3/2 8 / 51