16.322 Stochastic Estimation and Control, Fall 2004 Prof vander velde The second is the x which maximizes f(x|=,,三) f(x,,,) f(…,,1x)=/(=”,=,x) f(x) f(x|=,=) =3…:二N f(x) The a priori distribution f(x), and the distribution of x is also involved in the joint distribution of the observations. Only if the distribution of x is flat for all x can we guarantee that these two functions will have maxima at the same value of x. But a flat distribution for x for all x is exactly the case when there is no a priori knowledge about x. In that case all values of x have equal probability density which is in fact an infinitesimal. We can consider f(x) in that case to be the limit of almost any convenient distribution as the variance>0o im a imG→∞ If, on the other hand, we do have some prior information about x based on some physical reason, f(x) will have some finite sha often a normal shape -and the x which maximizes f(==xlx)will not maximize f(x|=1-x).In is case the latter choice of x is to be preferred since it is the most probable value of x based on the a priori distribution of x and the values of the observations whereas the former depends only on the observations We just note that for ther distributie which is symmetric about the maximum point, the most probable value is equal to the conditional mean which is the minimum variance value of x. in the absence of prior information, this is also the maximum likelihood estimate, which is the least weighted squares estimate. This we found to be a linear combination of the data. So in the case of a normal conditional distribution with no prior information, the optimum linear estimate based on the data is the minimum variance estimator. No nonlinear operation on the data can give a smaller variance estimate ge 3 of 816.322 Stochastic Estimation and Control, Fall 2004 Prof. Vander Velde Page 3 of 8 The second is the x which maximizes ( ) ( ) ( ) ( ) ( ) ( ) ( ) 1 1 1 1 1 1 1 , ,..., | ,..., ,..., ,..., , ,..., | ( ) ,..., | ,..., ( ) N N N N N N N f xz z f xz z fz z fz z x fz z x f x f z z f xz z f x = = = The a priori distribution f ( ) x , and the distribution of x is also involved in the joint distribution of the observations. Only if the distribution of x is flat for all x can we guarantee that these two functions will have maxima at the same value of x . But a flat distribution for x for all x is exactly the case when there is no a priori knowledge about x . In that case all values of x have equal probability density which is in fact an infinitesimal. We can consider f ( ) x in that case to be the limit of almost any convenient distribution as the variance → ∞ . If, on the other hand, we do have some prior information about x based on some previous measurements or on physical reason, f ( ) x will have some finite shape – often a normal shape – and the x which maximizes f (z zx 1,..., | N ) will not maximize ( ) 1 | ,..., N f xz z . In this case the latter choice of x is to be preferred since it is the most probable value of x based on the a priori distribution of x and the values of the observations, whereas the former depends only on the observations. We just note in passing that for a normal ( ) 1 | ,..., N f xz z or any other distribution which is symmetric about the maximum point, the most probable value is equal to the conditional mean which is the minimum variance value of xˆ . In the absence of prior information, this is also the maximum likelihood estimate, which is the least weighted squares estimate. This we found to be a linear combination of the data. So in the case of a normal conditional distribution with no prior information, the optimum linear estimate based on the data is the minimum variance estimator. No nonlinear operation on the data can give a smaller variance estimate