法国数学家勒让德于1806年首次发表最小二乘理论。 德国的高斯于1794年已经应用这一理论推算了谷神星的轨 道，但迟至1809年才正式发表。 CH6 LS Method RLS Algorithm RLS procedure is derived from least squares estimation theory. The LMS algorithm may then be viewed as a pruned version of the RLS algorithm. In contrast to the usual approach in which the LMS algorithm is derived from Wiener filter theory,based on steepest-descent optimization and instantaneous estimates of the process statistics
CH6 LS Method & RLS Algorithm RLS procedure is derived from least squares estimation theory. The LMS algorithm may then be viewed as a pruned version of the RLS algorithm. In contrast to the usual approach in which the LMS algorithm is derived from Wiener filter theory, based on steepest-descent optimization and instantaneous estimates of the process statistics. 法国数学家勒让德于1806年首次发表最小二乘理论。 德国的高斯于1794年已经应用这一理论推算了谷神星的轨 道，但迟至1809年才正式发表
Contents o S1.Introduction o S2.LS Method o S3.RLS algorithm o S4.Examples 2020-01-18 2
2020-01-18 2 Contents S1. Introduction S2. LS Method S3. RLS algorithm S4. Examples
S1.Introduction o 'given statistics'case Wiener filter theory:Probabilistic cost function statistical information on the stochastic processes involved is available. The Wiener-Hopf equations may be solved if the correlation matrix and cross-correlation vector are given. o‘given data'case In most applications,however,only data sequences are given,so that the process statistics have to be estimated from these data. Data based cost function. 2020-01-18 3
2020-01-18 3 S1. Introduction ‘given statistics’ case Wiener filter theory：Probabilistic cost function statistical information on the stochastic processes involved is available. The Wiener-Hopf equations may be solved if the correlation matrix and cross-correlation vector are given. ‘given data’ case In most applications, however, only data sequences are given, so that the process statistics have to be estimated from these data. Data based cost function
Self-designing filter o The filter is supplemented with an adaptation algorithm, ● monitor the environment (process statistics) vary the filter transfer function accordingly. 2020-01-18 4
2020-01-18 4 Self-designing filter The filter is supplemented with an adaptation algorithm, monitor the environment (process statistics) vary the filter transfer function accordingly
Detour:probabilistic machinery 0 In the 'given data'case,time averaging represents a practical means for the estimation of the process statistics (after invoking stationarity and ergodicity). o RLS algorithm may be derived starting from Wiener filter theory and employing time averaged estimates of the statistical parameters. o Viewing signals as realizations of stochastic processes,and then trying to estimate the corresponding process statistics is somewhat of a detour,which to some extent can be avoided. 2020-01-18 5
2020-01-18 5 Detour: probabilistic machinery In the ‘given data’ case, time averaging represents a practical means for the estimation of the process statistics (after invoking stationarity and ergodicity). RLS algorithm may be derived starting from Wiener filter theory and employing time averaged estimates of the statistical parameters. Viewing signals as realizations of stochastic processes, and then trying to estimate the corresponding process statistics is somewhat of a detour, which to some extent can be avoided
Direct approach o Data based cost function. A valid alternative to the probabilistic detour leads to comparable results (e.g.the RLS procedure) without having to rely on all the probabilistic machinery (stationarity, ergodicity,etc.) 2020-01-18 6
2020-01-18 6 Direct approach Data based cost function. A valid alternative to the probabilistic detour leads to comparable results (e.g. the RLS procedure) without having to rely on all the probabilistic machinery (stationarity, ergodicity, etc.)
True adaptive filtering O F Batch-mode processing ● a complete batch of data is available to design the optimal filter (i.e.the least squares parameter estimation problem). o True adaptive filtering:RLS The algorithm starts from a set of initial conditions,which may correspond to complete ignorance about the environment, ● And then adapts itself while doing the filtering operation. 2020-01-18 7
2020-01-18 7 True adaptive filtering Batch-mode processing a complete batch of data is available to design the optimal filter (i.e. the least squares parameter estimation problem). True adaptive filtering: RLS The algorithm starts from a set of initial conditions, which may correspond to complete ignorance about the environment, And then adapts itself while doing the filtering operation
S2.LS Method o The basic set-up of the filter o Data based cost function o LS estimation versus WF design o LS versus Orthogonal Principle o The sufficient-order problem o SVD solution o WLS,TLS,IRWLS... 2020-01-18 8
2020-01-18 8 S2. LS Method The basic set-up of the filter Data based cost function LS estimation versus WF design LS versus Orthogonal Principle The sufficient-order problem SVD solution WLS, TLS, IRWLS…
(1)basic set-up of the filter o Similar to WF o But the filter is fed by true data sequences instead of stochastic processes o The filter is fed by an input sequence either the FIR case or the linear combiner case o The error signal is e欧=d-w吸=康-ufW 1≤k≤L. 2020-01-18 9
2020-01-18 9 (1) basic set-up of the filter Similar to WF But the filter is fed by true data sequences instead of stochastic processes The filter is fed by an input sequence either the FIR case or the linear combiner case The error signal is
Matrix form ey d 吲 e d 吗 M 三 : : WN-1 eL dL 吲 W d U 2020-01-18 10
2020-01-18 10 Matrix form