Estimation Theory Least Square (LS) Wenhui Xiong NCL UESTC whxiong@uestc.edu.cn
whxiong@uestc.edu.cn Estimation Theory Least Square (LS) Wenhui Xiong NCL UESTC
Review MVU Estimator CRLB→factorize alnp(x;) =I(0)(g(x)-8) 80 Linear Model easy to find MVU RBLS: .Use sufficient statistics:T(x)ET(] unbiased estimator over sufficient statistics:g[T(x)] BLUE:if the unknown is a linear function of the data ●MLE:known PDF Minimize the estimator's variance whxiong@uestc.edu.cn 2
whxiong@uestc.edu.cn Review MVU Estimator 2 CRLB factorize Linear Model easy to find MVU RBLS: Use sufficient statistics: T(x) unbiased estimator over sufficient statistics: g[T(x)] BLUE: if the unknown is a linear function of the data MLE: known PDF E[µ · jT(x)] Minimize the estimator’s variance @lnp(x; µ) @µ = I(µ)(g(x) ¡ µ)
How about minimizing the square error? J=(n-sm)2 Mean square error(MSE) whxiong@uestc.edu.cn 3
whxiong@uestc.edu.cn 3 J = Mean square error (MSE) X n (x[n] ¡ s[n]) 2 How about minimizing the square error?
How about minimizing the square error? J=>(a[n]-s[nj)2 Mean square error(MSE) m Example:Estimating DC in white noise xln A+wln] whxiong@uestc.edu.cn 4
whxiong@uestc.edu.cn 4 J = Mean square error (MSE) X n (x[n] ¡ s[n]) 2 How about minimizing the square error? Example: Estimating DC in white noise x[n] = A + w[n]
How about minimizing the square error? J=>(x(n]-s[nl)2 Mean square error(MSE) m Example:Estimating DC in white noise xln A+wln] J(A)=(x[n)-A)2 Minimize J(A)w.r.t A whxiong@uestc.edu.cn 5
whxiong@uestc.edu.cn 5 J = Mean square error (MSE) X n (x[n] ¡ s[n]) 2 How about minimizing the square error? Minimize J(A) w.r.t A Example: Estimating DC in white noise x[n] = A + w[n] J(A) = X n (x[n]) ¡ A) 2
How about minimizing the square error? J=>(a[n]-s(n])2 Mean square error(MSE) m Example:Estimating DC in white noise xln A+wln] J(A)=(xm)-A)2 Minimize J(A)w.r.t A oA-[∑训-v=0→i-发∑m whxiong@uestc.edu.cn 6
whxiong@uestc.edu.cn 6 J = Mean square error (MSE) X n (x[n] ¡ s[n]) 2 How about minimizing the square error? Minimize J(A) w.r.t A Example: Estimating DC in white noise x[n] = A + w[n] J(A) = X n (x[n]) ¡ A) 2 @J(A) @A = 2 hXx[n] ¡ NA i = 0 A b = 1 N X n x(n)
How about minimizing the square error? J=>(x(n]-s[nl)2 Mean square error(MSE) m Example:Estimating DC in white noise xln A+wln] J(A)=(xn)-A)2 Minimize J(A)w.r.t A oA-区-网-u→i-∑m Q:Is this estimator MVU? whxiong@uestc.edu.cn
whxiong@uestc.edu.cn 7 J = Mean square error (MSE) X n (x[n] ¡ s[n]) 2 How about minimizing the square error? Minimize J(A) w.r.t A Example: Estimating DC in white noise x[n] = A + w[n] J(A) = X n (x[n]) ¡ A) 2 @J(A) @A = 2 hXx[n] ¡ NA i = 0 A b = 1 N X n x(n) Q: Is this estimator MVU?
How about minimizing the square error? J=∑(xn-sn)2 Mean square error(MSE) Example:Estimating DC in white noise xln A+wln] J(A)=(xm)-A)2 Minimize J(A)w.r.t A o∑m-N个-0→1=∑w Q:Is this estimator MVU? A:yes,iff w[n]is AWGN E[A]=A+Elw(n)] whxiong@uestc.edu.cn 8
whxiong@uestc.edu.cn 8 J = Mean square error (MSE) X n (x[n] ¡ s[n]) 2 How about minimizing the square error? Minimize J(A) w.r.t A Example: Estimating DC in white noise x[n] = A + w[n] J(A) = X n (x[n]) ¡ A) 2 @J(A) @A = 2 hXx[n] ¡ NA i = 0 A b = 1 N X n x(n) Q: Is this estimator MVU? A: yes, iff w[n] is AWGN E[A b ] = A + E[w(n)]
Least Square (LS) Similar to linear model,assume s =I0 whxiong@uestc.edu.cn 9
whxiong@uestc.edu.cn Least Square (LS) 9 Similar to linear model, assume s = Hµ
Least Square (LS) Similar to linear model,assume s=H J()=∑(xln-s[n)2 whxiong@uestc.edu.cn 10
whxiong@uestc.edu.cn Least Square (LS) 10 Similar to linear model, assume J(µ) = X(x[n] ¡ s[n]) 2 s = Hµ
