Here,the first and last two equalities use the definition of the trace operator and matrix multiplication.The fourth equality,where the main work occurs,uses the commutativity of scalar multiplication in order to reverse the order of the terms in each product,and the commutativity and associativity of scalar addition in order to rearrange the order of the summation. 3.5 Norms A norm of a vector is informally a measure of the "length"of the vector.For example, we have the commonly-used Euclidean or e2 norm, 2 Note that竖=xTx. More formally,a norm is any function f:R"-R that satisfies 4 properties: l.For all x∈Rn,f(x)≥0(non-negativity) 2.f(r)=0 if and only if x =0 (definiteness). 3.For all xE R",tER,f(tx)=tf(r)(homogeneity) 4.For all,y∈R",f(x+y）≤f(r)+f(y)）(triangle inequality). Other examples of norms are the ei norm, lh=∑zl and the loo norm, llclloo maxiil. In fact,all three norms presented so far are examples of the family of ep norms,which are parameterized by a real number p>1,and defined as 1 Norms can also be defined for matrices,such as the Frobenius norm, A号=V(AT④. Many other norms exist,but they are beyond the scope of this review. 10Here, the first and last two equalities use the definition of the trace operator and matrix multiplication. The fourth equality, where the main work occurs, uses the commutativity of scalar multiplication in order to reverse the order of the terms in each product, and the commutativity and associativity of scalar addition in order to rearrange the order of the summation. 3.5 Norms A norm of a vector kxk is informally a measure of the “length” of the vector. For example, we have the commonly-used Euclidean or ℓ2 norm, kxk2 = vuutXn i=1 x 2 i . Note that kxk 2 2 = x T x. More formally, a norm is any function f : R n → R that satisfies 4 properties: 1. For all x ∈ R n , f(x) ≥ 0 (non-negativity). 2. f(x) = 0 if and only if x = 0 (definiteness). 3. For all x ∈ R n , t ∈ R, f(tx) = |t|f(x) (homogeneity). 4. For all x, y ∈ R n , f(x + y) ≤ f(x) + f(y) (triangle inequality). Other examples of norms are the ℓ1 norm, kxk1 = Xn i=1 |xi | and the ℓ∞ norm, kxk∞ = maxi |xi |. In fact, all three norms presented so far are examples of the family of ℓp norms, which are parameterized by a real number p ≥ 1, and defined as kxkp = Xn i=1 |xi | p !1/p . Norms can also be defined for matrices, such as the Frobenius norm, kAkF = vuut Xm i=1 Xn j=1 A2 ij = p tr(ATA). Many other norms exist, but they are beyond the scope of this review. 10