15.7 Robust Estimation 701 where the function p is the negative logarithm of the probability density.Taking the logarithm of(15.7.1)analogously with(15.1.4),we find that we want to minimize the expression p(vi,y{xi;a}) (15.7.2) =1 Very often,it is the case that the function p depends not independently on its two arguments,measured yi and predicted y(i),but only on their difference,at least if scaled by some weight factorso;which we are able to assign to each point.In this 81 case the M-estimate is said to be local,and we can replace (15.7.2)by the prescription -y(xi;a minimize over ∑p( (15.7.3) i-I where the function p(z)is a function of a single variable z=[yi-y(i)]/oi. If we now define the derivative of p(z)to be a function () RECIPES ()d() 49 (15.7.4) dz Press. then the generalization of(15.1.7)to the case of a general M-estimate is N 0= ∑1 Oy(xi;a) k=1,,M Oak (15.7.5) 40i i= IENTIFIC If you compare (15.7.3)to (15.1.3),and (15.7.5)to (15.1.7),you see at once 6 that the specialization for normally distributed errors is p= (2)=2 (normal) (15.7.6) MPUTING (ISBN 1992 If the errors are distributed as a double or two-sided exponential,namely Prob yi-y(i)}~exp (15.7.7) Numerical 10521 431 then,by contrast, (outside Recipes p(x)=lz(z）=sgn(z) (double exponential) (15.7.8) North Software. Comparing to equation(15.7.3),we see that in this case the maximum likelihood estimator is obtained by minimizing the mean absolute deviation,rather than the mean square deviation.Here the tails of the distribution,although exponentially decreasing,are asymptotically much larger than any corresponding Gaussian. A distribution with even more extensive-therefore sometimes even more realistic -tails is the Cauchy or Lorentzian distribution. 1 Prob yi-y(i) (15.7.9) 1+15.7 Robust Estimation 701 Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machine￾readable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). where the function ρ is the negative logarithm of the probability density. Taking the logarithm of (15.7.1) analogously with (15.1.4), we find that we want to minimize the expression
N i=1 ρ(yi, y {xi; a}) (15.7.2) Very often, it is the case that the function ρ depends not independently on its two arguments, measured yi and predicted y(xi), but only on their difference, at least if scaled by some weight factors σi which we are able to assign to each point. In this case the M-estimate is said to be local, and we can replace (15.7.2) by the prescription minimize over a
N i=1 ρ yi − y(xi; a) σi  (15.7.3) where the function ρ(z) is a function of a single variable z ≡ [yi − y(xi)]/σi. If we now define the derivative of ρ(z) to be a function ψ(z), ψ(z) ≡ dρ(z) dz (15.7.4) then the generalization of (15.1.7) to the case of a general M-estimate is 0 =
N i=1 1 σi ψ yi − y(xi) σi  ∂y(xi; a) ∂ak  k = 1,...,M (15.7.5) If you compare (15.7.3) to (15.1.3), and (15.7.5) to (15.1.7), you see at once that the specialization for normally distributed errors is ρ(z) = 1 2 z2 ψ(z) = z (normal) (15.7.6) If the errors are distributed as a double or two-sided exponential, namely Prob {yi − y(xi)} ∼ exp −     yi − y(xi) σi      (15.7.7) then, by contrast, ρ(x) = |z| ψ(z) = sgn(z) (double exponential) (15.7.8) Comparing to equation (15.7.3), we see that in this case the maximum likelihood estimator is obtained by minimizing the mean absolute deviation, rather than the mean square deviation. Here the tails of the distribution, although exponentially decreasing, are asymptotically much larger than any corresponding Gaussian. A distribution with even more extensive — therefore sometimes even more realistic — tails is the Cauchy or Lorentzian distribution, Prob {yi − y(xi)} ∼ 1 1 + 1 2 yi − y(xi) σi 2 (15.7.9)