Performance of optical flow Barron, fleet and beauchemin JcV12:1,1994 http://www.csd.uwoca/faculty/barron/
Performance of Optical Flow Barron, Fleet and Beauchemin IJCV 12:1, 1994 http://www.csd.uwo.ca/faculty/barron/
Performance of optical flow Evaluation of different optical flow techniques Accuracy reliability density of measurements a common set of synthetic and real sequences Several optical flow methods Differential Matching Energy-based Phase-based
Performance of Optical Flow • Evaluation of different optical flow techniques – Accuracy, reliability, density of measurements • A common set of synthetic and real sequences • Several optical flow methods – Differential – Matching – Energy-based – Phase-based
Performance of optical flow Accurate and dense velocity measurement Accurate 2d motion filed estimation is ill posed Inherent differences between the 2d motion field and intensity variations Only qualitative information can be extracted
Performance of Optical Flow • Accurate and dense velocity measurement • Accurate 2d motion filed estimation is illposed – Inherent differences between the 2D motion field and intensity variations • Only qualitative information can be extracted
Optical flow Process · Three stages Perfiltering or smoothing with low-pass /band-pass filters in order to extract signal structure of interest enhance the signal-to-noise ratio Extraction of basic measurements Spatiotemporal derivatives Local correlation surface Integration of measurements to produce 2D flow field Often involves assumptions about the smoothness of the underlying flow field
Optical Flow Process • Three stages – Perfiltering or smoothing with low-pass/band-pass filters in order to • extract signal structure of interest • enhance the signal-to-noise ratio – Extraction of basic measurements • Spatiotemporal derivatives • Local correlation surface – Integration of measurements to produce 2D flow field • Often involves assumptions about the smoothness of the underlying flow field
Differential Techniques First-order derivatives and based on image translation /(x,t)=l(X-v1,0 Intensity Is conserved d(r, t) 0 t VI(x,1)v+1(x,)=0V/(x,)=(1(x,),(x1) Normal velocity s(x,) 1, (x, t) V(x, t) n=sn n(x VI(x, t)I
Differential Techniques • First-order derivatives and based on image translation • Intensity is conserved • Normal velocity I t I t ( , ) ( ,0) x x v = − ( , )T v = u v ( , ) 0 dI t dt = x ( , ) ( , ) 0 t + = I t I t x v x ( , ) ( ( , ), ( , ))T x y = I t I t I t x x x n v n = s ( , ) ( , ) ( , ) t I t s t I t − = x x x ( , ) ( , ) ( , ) t t I t = x n x x
Differential Techniques Second-order differential In (x, t)I(x, On./Iu(x, l(x,)n(x,) xt 0 Stronger restriction than first-order derivatives on permissible motion field VI(x.,1)v+l1(x,)=0 Can be combined with 1st order in isolation or together(over determined system) Velocity estimation from 2nd-order methods are often assumed be to sparser and less accurate than estimation from 1st-order methods
Differential Techniques • Second-order differential • Stronger restriction than first-order derivatives on permissible motion field • Can be combined with 1st order in isolation or together (overdetermined system) • Velocity estimation from 2nd -order methods are often assumed be to sparser and less accurate than estimation from 1st -order methods 1 2 ( , ) ( , ) ( , ) 0 ( , ) ( , ) ( , ) 0 xx yx tx xy yy tx I t I t v I t I t I t v I t + = x x x x x x ( , ) ( , ) 0 t + = I t I t x v x
Differential Techniques Additional constraints Fits the measurements in each neighborhood to a local model for 2d velocity Using least squares minimization or Hough transform Global smoothness
Differential Techniques • Additional constraints – Fits the measurements in each neighborhood to a local model for 2d velocity • Using least squares minimization or Hough transform – Global smoothness
Differential Techniques I(x, t)must be differentiable Temporal smoothing at the sensors is needed to avoid aliasing Numerical differentiation must be done carefully If aliasing can not be avoided in image acquisition apply differential techniques in a coarse-to-fine manner
Differential Techniques • must be differentiable – Temporal smoothing at the sensors is needed to avoid aliasing – Numerical differentiation must be done carefully • If aliasing can not be avoided in image acquisition – Apply differential techniques in a coarse-to-fine manner I t ( , ) x
Horn and schunck Combine gradient constraint with a global smoothness term, minimizing 「(Vv+4)2+(+)k n=0.5 instead of 2=100 Ⅰ(Ⅰv+Ⅰ uk+=ukx v=10=0 a2+2+ k+1 k(1n+1v+1) a2+2+Ⅰ
Horn and Schunck • Combine gradient constraint with a global smoothness term, minimizing ( ) ( ) 2 2 2 2 D t 2 2 + + + I I u v d v x 0 0 v u = = 0 1 222 1 222 ( ) ( ) k k k k x x y t x y k k k k y x y t x y I I u I v I u u I I I I u I v I v v I I + + + + = − + + + + = − + + = = 0.5 instead of 100
Horn and schunck Relatively crude form of numerical differentiation can be source of error Spatiotemporal smoothing Gaussian prefilter with o= 1.5 pixels in space and 1. 5 frames in time 4-point central differences for differentiation mask;(-1,8.0,-81)
Horn and Schunck • Relatively crude form of numerical differentiation can be source of error • Spatiotemporal smoothing – Gaussian prefilter with 1.5 pixels in space and 1.5 frames in time • 4-point central differences for differentiation – mask 1 ( 1,8,0, 8,1) 12 − − =