AsySVRG Lock-free Analysis In all the GD or SGD methods to solve the objective function,the key step can be written as u←-u+△ Notation Aij:thejth update vector computed by the iththread; 。u E Rd and u=(u),u(②)，，ud)i 。ty:the time when the operation"←u+△has been completed(Not the time when the operation begins); Assuming i,j.which can be easily guaranteed by programming 4口，4@下424，分QC Wu-Jun Li (http://cs.nju.edu.cn/lwj) PDSL CS.NJU 11 /36AsySVRG Lock-free Analysis In all the GD or SGD methods to solve the objective function, the key step can be written as u ← u + ∆ Notation ∆i,j : the j th update vector computed by the i th thread; u ∈ R d and u = (u (1), u(2), . . . , u(d) ); t (k) i,j : the time when the operation “u (k) ← u (k) + ∆(k) i,j ” has been completed (Not the time when the operation begins); Assuming ∀i, j, t (1) i,j < t(2) i,j < . . . < t(d) i,j , which can be easily guaranteed by programming Wu-Jun Li (http://cs.nju.edu.cn/lwj) PDSL CS, NJU 11 / 36