Statistical Learning Theory and Applications Lecture 2 Review of Linear Algebra and Probability Theory Instructor:Quan Wen SCSE@UESTC Fall 2021
Statistical Learning Theory and Applications Lecture 2 Review of Linear Algebra and Probability Theory Instructor: Quan Wen SCSE@UESTC Fall 2021
Outline (Level 1) ①Linear Algebra ②Probability Theory 1/130
Outline (Level 1) 1 Linear Algebra 2 Probability Theory 1 / 130
Topics: Basic concepts,properties and calculations of vectors o Basic concepts,properties and calculations of matrices o Basic concepts,properties and calculations of probability Basic concepts,properties and calculations of random variable and it's distributions Common probability distributions Key points and difficulties: o Key points:Main concepts and properties of linear algebra and probability theory,and common probability distributions involved in statistical learning Difficulties:Concept and understanding of random variables 2/130
Topics: Basic concepts, properties and calculations of vectors Basic concepts, properties and calculations of matrices Basic concepts, properties and calculations of probability Basic concepts, properties and calculations of random variable and it’s distributions Common probability distributions Key points and difficulties: Key points: Main concepts and properties of linear algebra and probability theory, and common probability distributions involved in statistical learning. Difficulties: Concept and understanding of random variables 2 / 130
Outline (Level 1) ○Linear Algebra Probability Theory 3/130
Outline (Level 1) 1 Linear Algebra 2 Probability Theory 3 / 130
1.Linear Algebra Linear algebra provides a way of compactly representing and operating on sets of linear equations.For example,consider the following system of equations: 4x1-5x2=-13 -2x1+3x2=9 In matrix notation. Ax=b where 4=()=()6=() 4/130
1. Linear Algebra Linear algebra provides a way of compactly representing and operating on sets of linear equations. For example, consider the following system of equations: 4x1 − 5x2 = −13 −2x1 + 3x2 = 9 In matrix notation, Ax = b where A = 4 −5 −2 3 , x = x1 x2 , b = −13 9 4 / 130
Outline (Level 1-2) ○Linear Algebra o Notations and Fundamental Concepts o Matrix Multiplication o Operational Properties o Linear Space o Quadratic Form and Positive Definite Matrix o Matrix Calculus 5/130
Outline (Level 1-2) 1 Linear Algebra Notations and Fundamental Concepts Matrix Multiplication Operational Properties Linear Space Quadratic Form and Positive Definite Matrix Matrix Calculus 5 / 130
Outline (Level 2-3) Notations and Fundamental Concepts o Matrix and Vector 6/130
Outline (Level 2-3) Notations and Fundamental Concepts Matrix and Vector 6 / 130
1.1.Notations and Fundamental Concepts 1.1.1.Matrix and Vector Matrix and Transpose:A E Rx",m rows,n columns d11 a12 d13 ain d21 a22 a23 a2n A= aml am2 am3 amn d11 a21 a31 aml AT= d12 a22 a32 am2 ain a2n a3n amn 7/130
1.1. Notations and Fundamental Concepts 1.1.1. Matrix and Vector Matrix and Transpose: A ∈ R m×n , m rows, n columns A = a11 a12 a13 · · · a1n a21 a22 a23 · · · a2n . . . . . . . . . . . . . . . am1 am2 am3 · · · amn A T = a11 a21 a31 · · · am1 a12 a22 a32 · · · am2 . . . . . . . . . . . . . . . a1n a2n a3n · · · amn 7 / 130
Vector and Matrix:d-dimensional column vector x and its transpose x'or x: X1 X2 X= ,x=X=(1,x2,…,xa Xd Denote the j-th column of A by aj,4..or A.,j,a is column vector. A=[a1,a2,..,an. Denote the i-th row of A by af,4i.:or 4i.,af is row vector.Can be understand as the row version of the i-th column of 4 A= a】 8/130
Vector and Matrix: d-dimensional column vector x and its transpose x ′ or x T : x = x1 x2 . . . xd , x T = x ′ = (x1, x2, · · · , xd) ▶ Denote the j-th column of A by aj , A:, j or A∗, j , aj is column vector. A = [a1, a2, · · · , an] . ▶ Denote the i-th row of A by a T i , Ai,: or Ai,∗, a T i is row vector. Can be understand as the row version of the i-th column of A T A = a T 1 a T 2 . . . a T m 8 / 130
Outline (Level 1-2) ○Linear Algebra o Notations and Fundamental Concepts Matrix Multiplication o Operational Properties o Linear Space o Quadratic Form and Positive Definite Matrix o Matrix Calculus 9/130
Outline (Level 1-2) 1 Linear Algebra Notations and Fundamental Concepts Matrix Multiplication Operational Properties Linear Space Quadratic Form and Positive Definite Matrix Matrix Calculus 9 / 130