532 Chapter 12.Fast Fourier Transform 12.6 External Storage or Memory-Local FFTs Sometime in your life,you might have to compute the Fourier transform of a really large data set,larger than the size of your computer's physical memory.In such a case, the data will be stored on some external medium,such as magnetic or optical tape or disk. Needed is an algorithm that makes some manageable number of sequential passes through the external data,processing it on the fly and outputting intermediate results to other external media,which can be read on subsequent passes. In fact,an algorithm of just this description was developed by Singleton [1]very soon after the discovery of the FFT.The algorithm requires four sequential storage devices,each capable of holding half of the input data.The first half of the input data is initially on one 8 device,the second half on another Singleton's algorithm is based on the observation that it is possible to bit-reverse2 values by the following sequence of operations:On the first pass,values are read alternately from the two input devices,and written to a single output device (until it holds half the data), and then to the other output device.On the second pass,the output devices become input devices,and vice versa.Now,we copy nwo values from the first device,then two values from the second,writing them (as before)first to fill one output device,then to fill a second. Subsequent passes read 4,8,etc.,input values at a time.After completion of pass M-1, ⊙ the data are in bit-reverse order. Singleton's next observation is that it is possible to alternate the passes of essentially this bit-reversal technique with passes that implement one stage of the Danielson-Lanczos combination formula (12.2.3).The scheme,roughly,is this:One starts as before with half 导不 Press. the input data on one device,half on another.In the first pass,one complex value is read ART from each input device.Two combinations are formed,and one is written to each of two output devices.After this“computing”pass,the devices are rewound,anda“permutation” Programs pass is performed,where groups of values are read from the first input device and alternately written to the first and second output devices;when the first input device is exhausted,the second is similarly processed.This sequence of computing and permutation passes is repeated SCIENTIFIC M-K-1 times,where 2 is the size of internal buffer available to the program.The second phase of the computation consists of a final K computation passes.What distinguishes the second phase from the first is that,now,the permutations are local enough to do in place during the computation.There are thus no separate permutation passes in the second phase. In all,there are 2M-K-2 passes through the data. Here is an implementation of Singleton's algorithm,based on[1]: (ISBN #include <stdio.h> 21 #include <math.h> #include "nrutil.h" #define KBF 128 Numerical Recipes 43108 void fourfs(FILE *file[5],unsigned long nn[],int ndim,int isign) One-or multi-dimensional Fourier transform of a large data set stored on external media.On (outside input,ndim is the number of dimensions,and nn[1..ndim]contains the lengths of each di- mension (number of real and imaginary value pairs),which must be powers of two.file[1..4] North Software. contains the stream pointers to 4 temporary files,each large enough to hold half of the data. The four streams must be opened in the system's "binary"(as opposed to "text")mode.The input data must be in C normal order,with its first half stored in file file[1],its second half in file[2],in native floating point form.KBF real numbers are processed per buffered read or write.isign should be set to 1 for the Fourier transform,to -1 for its inverse.On output,values in the array file may have been permuted;the first half of the result is stored in file [3],the second half in file [4].N.B.:For ndim 1,the output is stored by columns, i.e.,not in C normal order:in other words,the output is the transpose of that which would have been produced by routine fourn. void fourew(FILE *file[5],int *na,int *nb,int *nc,int *nd); unsigned long j,j12,jk,k,kk,n=1,mm,kc=0,kd,ks,kr,nr,ns,nv; int cc,na,nb,nc,nd;532 Chapter 12. Fast Fourier Transform Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machine￾readable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). 12.6 External Storage or Memory-Local FFTs Sometime in your life, you might have to compute the Fourier transform of a really large data set, larger than the size of your computer’s physical memory. In such a case, the data will be stored on some external medium, such as magnetic or optical tape or disk. Needed is an algorithm that makes some manageable number of sequential passes through the external data, processing it on the fly and outputting intermediate results to other external media, which can be read on subsequent passes. In fact, an algorithm of just this description was developed by Singleton [1] very soon after the discovery of the FFT. The algorithm requires four sequential storage devices, each capable of holding half of the input data. The first half of the input data is initially on one device, the second half on another. Singleton’s algorithm is based on the observation that it is possible to bit-reverse 2M values by the following sequence of operations: On the first pass, values are read alternately from the two input devices, and written to a single output device (until it holds half the data), and then to the other output device. On the second pass, the output devices become input devices, and vice versa. Now, we copy two values from the first device, then two values from the second, writing them (as before) first to fill one output device, then to fill a second. Subsequent passes read 4, 8, etc., input values at a time. After completion of pass M − 1, the data are in bit-reverse order. Singleton’s next observation is that it is possible to alternate the passes of essentially this bit-reversal technique with passes that implement one stage of the Danielson-Lanczos combination formula (12.2.3). The scheme, roughly, is this: One starts as before with half the input data on one device, half on another. In the first pass, one complex value is read from each input device. Two combinations are formed, and one is written to each of two output devices. After this “computing” pass, the devices are rewound, and a “permutation” pass is performed, where groups of values are read from the first input device and alternately written to the first and second output devices; when the first input device is exhausted, the second is similarly processed. This sequence of computing and permutation passes is repeated M − K − 1 times, where 2K is the size of internal buffer available to the program. The second phase of the computation consists of a final K computation passes. What distinguishes the second phase from the first is that, now, the permutations are local enough to do in place during the computation. There are thus no separate permutation passes in the second phase. In all, there are 2M − K − 2 passes through the data. Here is an implementation of Singleton’s algorithm, based on [1]: #include <stdio.h> #include <math.h> #include "nrutil.h" #define KBF 128 void fourfs(FILE *file[5], unsigned long nn[], int ndim, int isign) One- or multi-dimensional Fourier transform of a large data set stored on external media. On input, ndim is the number of dimensions, and nn[1..ndim] contains the lengths of each di￾mension (number of real and imaginary value pairs), which must be powers of two. file[1..4] contains the stream pointers to 4 temporary files, each large enough to hold half of the data. The four streams must be opened in the system’s “binary” (as opposed to “text”) mode. The input data must be in C normal order, with its first half stored in file file[1], its second half in file[2], in native floating point form. KBF real numbers are processed per buffered read or write. isign should be set to 1 for the Fourier transform, to −1 for its inverse. On output, values in the array file may have been permuted; the first half of the result is stored in file[3], the second half in file[4]. N.B.: For ndim > 1, the output is stored by columns, i.e., not in C normal order; in other words, the output is the transpose of that which would have been produced by routine fourn. { void fourew(FILE *file[5], int *na, int *nb, int *nc, int *nd); unsigned long j,j12,jk,k,kk,n=1,mm,kc=0,kd,ks,kr,nr,ns,nv; int cc,na,nb,nc,nd;