13.3 Optimal(Wiener)Filtering with the FFT 547 13.3 Optimal (Wiener)Filtering with the FFT There are a number of other tasks in numerical processing that are routinely handled with Fourier techniques.One of these is filtering for the removal of noise from a"corrupted"signal.The particular situation we consider is this:There is some underlying,uncorrupted signal u(t)that we want to measure.The measurement process is imperfect,however,and what comes out of our measurement device is a corrupted signal c(t).The signal c(t)may be less than perfect in either or both of two respects.First,the apparatus may not have a perfect"delta-function"response, so that the true signal u(t)is convolved with(smeared out by)some known response function r(t)to give a smeared signal s(t), s(t)= r(t-T)u(T)dr or S(f)=R(f)U(f) (13.3.1) 蓉 where S,R,U are the Fourier transforms of s,r,u,respectively.Second,the measured signal c(t)may contain an additional component of noise n(t), RECIPES 9 c(t)=s(t)+n(t) (13.3.2) Press. We already know how to deconvolve the effects of the response function r in the absence of any noise($13.1);we just divide C(f)by R(f)to get a deconvolved signal.We now want to treat the analogous problem when noise is present.Our task is to find the optimal filter,o(t)or (f),which,when applied to the measured 色g台 signal c(t)or C(f),and then deconvolved by r(t)or R(f),produces a signal u(t) OF SCIENTIFIC( or U(f)that is as close as possible to the uncorrupted signal u(t)or U(f).In other words we will estimate the true signal U by 6 0U)=Cf)Φf) (13.3.3) R(f) In what sense is U to be close to U?We ask that they be close in the 10.621 least-square sense Numerica 431 )-vp a=-ud is minimized. (13.3.4) (outside Recipes Substituting equations(13.3.3)and (13.3.2),the right-hand side of (13.3.4)becomes North [S(f)+N(f)(f)s(f) R(f) R(f) d (13.3.5) IRf12{Sf)PI1-Φ(f)P+1NfI21()P}d西 The signal S and the noise N are uncorrelated,so their cross product,when integrated over frequency f,gave zero.(This is practically the definition of what we mean by noise!)Obviously (13.3.5)will be a minimum if and only if the integrand is minimized with respect to (f)at every value of f.Let us search for such a
13.3 Optimal (Wiener) Filtering with the FFT 547 Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machinereadable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). 13.3 Optimal (Wiener) Filtering with the FFT There are a number of other tasks in numerical processing that are routinely handled with Fourier techniques. One of these is filtering for the removal of noise from a “corrupted” signal. The particular situation we consider is this: There is some underlying, uncorrupted signal u(t) that we want to measure. The measurement process is imperfect, however, and what comes out of our measurement device is a corrupted signal c(t). The signal c(t) may be less than perfect in either or both of two respects. First, the apparatus may not have a perfect “delta-function” response, so that the true signal u(t) is convolved with (smeared out by) some known response function r(t) to give a smeared signal s(t), s(t) = ∞ −∞ r(t − τ)u(τ) dτ or S(f) = R(f)U(f) (13.3.1) where S, R, U are the Fourier transforms of s, r, u, respectively. Second, the measured signal c(t) may contain an additional component of noise n(t), c(t) = s(t) + n(t) (13.3.2) We already know how to deconvolve the effects of the response function r in the absence of any noise (§13.1); we just divide C(f) by R(f) to get a deconvolved signal. We now want to treat the analogous problem when noise is present. Our task is to find the optimal filter, φ(t) or Φ(f), which, when applied to the measured signal c(t) or C(f), and then deconvolved by r(t) or R(f), produces a signal u(t) or U(f) that is as close as possible to the uncorrupted signal u(t) or U(f). In other words we will estimate the true signal U by U(f) = C(f)Φ(f) R(f) (13.3.3) In what sense is U to be close to U? We ask that they be close in the least-square sense ∞ −∞ |u(t) − u(t)| 2 dt = ∞ −∞ U(f) − U(f) 2 df is minimized. (13.3.4) Substituting equations (13.3.3) and (13.3.2), the right-hand side of (13.3.4) becomes ∞ −∞ [S(f) + N(f)]Φ(f) R(f) − S(f) R(f) 2 df = ∞ −∞ |R(f)| −2 |S(f)| 2 |1 − Φ(f)| 2 + |N(f)| 2 |Φ(f)| 2 df (13.3.5) The signal S and the noise N are uncorrelated, so their cross product, when integrated over frequency f, gave zero. (This is practically the definition of what we mean by noise!) Obviously (13.3.5) will be a minimum if and only if the integrand is minimized with respect to Φ(f) at every value of f. Let us search for such a
548 Chapter 13.Fourier and Spectral Applications solution where (f)is a real function.Differentiating with respect to and setting the result equal to zero gives Φ(f)= 1S(f)2 Is(f)2+IN()2 (13.3.6) This is the formula for the optimal filter (f). Notice that equation (13.3.6)involves S,the smeared signal,and N,the noise. The two of these add up to be C,the measured signal.Equation(13.3.6)does not contain U,the"true"signal.This makes for an important simplification:The optimal filter can be determined independently of the determination of the deconvolution function that relates S and U. To determine the optimal filter from equation(13.3.6)we need some way of separately estimating S and N.There is no way to do this from the measured signal C alone without some other information,or some assumption or guess.Luckily,the extra information is often easy to obtain.For example,we can sample a long stretch of data c(t)and plot its power spectral density using equations(12.0.14),(12.1.8),and (12.1.5).This quantity is proportional to the sum 2 S+N,so we have |S(f)2+lN(f)2≈P.(f)=IC(f)20≤f<fe (13.3.7) (More sophisticated methods of estimating the power spectral density will be discussed in 813.4 and 813.7.but the estimation above is almost always good enough 06263 for the optimal filter problem.)The resulting plot (see Figure 13.3.1)will often immediately show the spectral signature of a signal sticking up above a continuous noise spectrum.The noise spectrum may be flat,or tilted,or smoothly varying;it doesn't matter,as long as we can guess a reasonable hypothesis as to what it is. Draw a smooth curve through the noise spectrum.extrapolating it into the region dominated by the signal as well.Now draw a smooth curve through the signal plus noise power.The difference between these two curves is your smooth"model"of the 喻 signal power.The quotient of your model of signal power to your model of signal Numerica 10621 plus noise power is the optimal filter (f).[Extend it to negative values of f by the formula (-f)=(f).]Notice that (f)will be close to unity where the noise is negligible,and close to zero where the noise is dominant.That is how it does its job!The intermediate dependence given by equation(13.3.6)just turns out to be the optimal way of going in between these two extremes. Because the optimal filter results from a minimization problem,the quality of the results obtained by optimal filtering differs from the true optimum by an amount that is secondorder in the precision to which the optimal filter is determined.In other words,even a fairly crudely determined optimal filter(sloppy,say,at the 10 percent level)can give excellent results when it is applied to data.That is why the separation of the measured signal C into signal and noise components S and N can usefully be done"by eye"from a crude plot of power spectral density.All of this may give you thoughts about iterating the procedure we have just described.For example,after designing a filter with response (f)and using it to make a respectable guess at the signal U(f)=(f)C(f)/R(f),you might turn about and regard U(f)as a fresh new signal which you could improve even further with the same filtering technique
548 Chapter 13. Fourier and Spectral Applications Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machinereadable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). solution where Φ(f) is a real function. Differentiating with respect to Φ, and setting the result equal to zero gives Φ(f) = |S(f)| 2 |S(f)| 2 + |N(f)| 2 (13.3.6) This is the formula for the optimal filter Φ(f). Notice that equation (13.3.6) involves S, the smeared signal, and N, the noise. The two of these add up to be C, the measured signal. Equation (13.3.6) does not containU, the “true” signal. This makes for an important simplification: The optimal filter can be determined independently of the determination of the deconvolution function that relates S and U. To determine the optimal filter from equation (13.3.6) we need some way of separately estimating |S| 2 and |N| 2 . There is no way to do this from the measured signal C alone without some other information, or some assumption or guess. Luckily, the extra information is often easy to obtain. For example, we can sample a long stretch of data c(t) and plot its power spectral density using equations (12.0.14), (12.1.8), and (12.1.5). This quantity is proportional to the sum |S| 2 + |N| 2 , so we have |S(f)| 2 + |N(f)| 2 ≈ Pc(f) = |C(f)| 2 0 ≤ f<fc (13.3.7) (More sophisticated methods of estimating the power spectral density will be discussed in §13.4 and §13.7, but the estimation above is almost always good enough for the optimal filter problem.) The resulting plot (see Figure 13.3.1) will often immediately show the spectral signature of a signal sticking up above a continuous noise spectrum. The noise spectrum may be flat, or tilted, or smoothly varying; it doesn’t matter, as long as we can guess a reasonable hypothesis as to what it is. Draw a smooth curve through the noise spectrum, extrapolating it into the region dominated by the signal as well. Now draw a smooth curve through the signal plus noise power. The difference between these two curves is your smooth “model” of the signal power. The quotient of your model of signal power to your model of signal plus noise power is the optimal filter Φ(f). [Extend it to negative values of f by the formula Φ(−f) = Φ(f).] Notice that Φ(f) will be close to unity where the noise is negligible, and close to zero where the noise is dominant. That is how it does its job! The intermediate dependence given by equation (13.3.6) just turns out to be the optimal way of going in between these two extremes. Because the optimal filter results from a minimization problem, the quality of the results obtained by optimal filtering differs from the true optimum by an amount that is second orderin the precision to which the optimal filter is determined. In other words, even a fairly crudely determined optimal filter (sloppy, say, at the 10 percent level) can give excellent results when it is applied to data. That is why the separation of the measured signal C into signal and noise components S and N can usefully be done “by eye” from a crude plot of power spectral density. All of this may give you thoughts about iterating the procedure we have just described. For example, after designing a filter with response Φ(f) and using it to make a respectable guess at the signal U(f) = Φ(f)C(f)/R(f), you might turn about and regard U(f) as a fresh new signal which you could improve even further with the same filtering technique.
13.4 Power Spectrum Estimation Using the FFT 549 IC2(measured) IN2(extrapolated) http://www.nr.com or call 1-800-872- Permission is read able files (including this one) granted fori IS2(deduced) internet Figure 13.3.1.Optimal (Wiener)filtering.The power spectrum of signal plus noise shows a signal peak to any server computer, from NUMERICAL RECIPES IN C: (North 1988-1992 by Cambridge University Press. added to a noise tail.The tail is extrapolated back into the signal region as a"noise model."Subtracting America tusers to make one paper THE gives the"signal model."The models need not be accurate for the method to be useful.A simple algebraic 是 ART combination of the models gives the optimal filter (see text). Programs Don't waste your time on this line of thought.The scheme converges to a signal of ictly proh S(f)=0.Converging iterative methods do exist;this just isn't one of them You can use the routine four1 (812.2)or realft (812.3)to FFT your data when you are constructing an optimal filter.To apply the filter to your data,you can use the methods described in $13.1.The specific routine convlv is not needed to dir for optimal filtering,since your filter is constructed in the frequency domain to begin with.If you are also deconvolving your data with a known response function, 1988-1992 however,you can modify convlv to multiply by your optimal filter just before it OF SCIENTIFIC COMPUTING(ISBN 0-521- takes the inverse Fourier transform. CITED REFERENCES AND FURTHER READING: Numerical Recipes -43108-5 Rabiner,L.R.,and Gold,B.1975,Theory and Application of Digital Signal Processing(Englewood Cliffs,NJ:Prentice-Hall). (outside Nussbaumer,H.J.1982,Fast Fourier Transform and Convolution Algorithms(New York:Springer- Verlag). North Software. Elliott,D.F.,and Rao.K.R.1982.Fast Transforms:Algorithms,Analyses,Applications (New York: Academic Press). visit website machine 13.4 Power Spectrum Estimation Using the FFT In the previous section we"informally"estimated the power spectral density ofa function c(t)by taking the modulus-squared of the discrete Fourier transform of some
13.4 Power Spectrum Estimation Using the FFT 549 Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machinereadable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). S 2 (deduced) N 2 (extrapolated) C 2 (measured) log scale f Figure 13.3.1. Optimal (Wiener) filtering. The power spectrum of signal plus noise shows a signal peak added to a noise tail. The tail is extrapolated back into the signal region as a “noise model.” Subtracting gives the “signal model.” The models need not be accurate for the method to be useful. A simple algebraic combination of the models gives the optimal filter (see text). Don’t waste your time on this line of thought. The scheme converges to a signal of S(f)=0. Converging iterative methods do exist; this just isn’t one of them. You can use the routine four1 (§12.2) or realft (§12.3) to FFT your data when you are constructing an optimal filter. To apply the filter to your data, you can use the methods described in §13.1. The specific routine convlv is not needed for optimal filtering, since your filter is constructed in the frequency domain to begin with. If you are also deconvolving your data with a known response function, however, you can modify convlv to multiply by your optimal filter just before it takes the inverse Fourier transform. CITED REFERENCES AND FURTHER READING: Rabiner, L.R., and Gold, B. 1975, Theory and Application of Digital Signal Processing (Englewood Cliffs, NJ: Prentice-Hall). Nussbaumer, H.J. 1982, Fast Fourier Transform and Convolution Algorithms (New York: SpringerVerlag). Elliott, D.F., and Rao, K.R. 1982, Fast Transforms: Algorithms, Analyses, Applications (New York: Academic Press). 13.4 Power Spectrum Estimation Using the FFT In the previous section we “informally” estimated the power spectral density of a function c(t) by taking the modulus-squared of the discrete Fourier transform of some