br=O(T ). Furture, if bT=O(T), then for every E >0, br =o(ra+).Wher br=o(T), it is simply (eventually) bounded and may or may not have a limit We often write O(1)in place of O(To). Similarly, br=o(1)means br=0 If each element of a vector or matrix is O(T )or o(T), then that vector or matrix is O(T )or o(Ta) Pr Let ar and br be scalar (i). If aT=O(T)and br=O(T"), then arbT=O(TAtH)and ar +bT=O(T), where=max{入,p (ii). If ar = o(T)and br= o(T"), then arbr= o(Tat) and ar +bT = o(T), where=max入,p (ii). If aT=O(T)and br =o(T), then arbr =o(rAt)and ar +bT=O(T) where=max入,p 1.2 Almost Sure Convergence The stochastic convergence concept most closely related to the limit notations previously discussed is that of almost sure convergence. Recall our discussing real-valued random variables br, we are in fact talking a mapping br: S-R. we let s be a typical element of sample space S, and call the real number br(s)a realization of the random variables Interest will often center on average such as br()=T1∑Z() t=1 Definition Let br(1 be a sequence of real-valued random variables. We say that br( converges almost surely to b, written br)- b if there exists a real number b such that PrIs: br(s-b=1. When no ambiguity is possible. we nay s imply write br-+b A sequence br converges almost surely if the probability of obtaining a realiza- tion of the sequence (Zt for which convergence to b occurs is unity. EquivalentlybT = O(T λ ). Furture, if bT = O(T λ ), then for every ξ > 0, bT = o(T λ+ξ ). When bT = O(T 0 ), it is simply (eventually) bounded and may or may not have a limit. We often write O(1) in place of O(T 0 ). Similarly, bT = o(1) means bT → 0. If each element of a vector or matrix is O(T λ ) or o(T λ ), then that vector or matrix is O(T λ ) or o(T λ ). Proposition: Let aT and bT be scalar. (i). If aT = O(T λ ) and bT = O(T µ ), then aT bT = O(T λ+µ ) and aT + bT = O(T κ ), where κ = max[λ, µ]. (ii). If aT = o(T λ ) and bT = o(T µ ), then aT bT = o(T λ+µ ) and aT + bT = o(T κ ), where κ = max[λ, µ]. (iii). If aT = O(T λ ) and bT = o(T µ ), then aT bT = o(T λ+µ ) and aT + bT = O(T κ ), where κ = max[λ, µ]. 1.2 Almost Sure Convergence The stochastic convergence concept most closely related to the limit notations previously discussed is that of almost sure convergence. Recall our discussing a real-valued random variables bT , we are in fact talking a mapping bT : S → R. we let s be a typical element of sample space S, and call the real number bT (s) a realization of the random variables. Interest will often center on average such as bT (·) = T −1X T t=1 Zt(·). Definition: Let {bT (·)} be a sequence of real-valued random variables. We say that bT (·) converges almost surely to b, written bT (·) a.s. −→ b if there exists a real number b such that Pr{s : bT (s) → b} = 1. When no ambiguity is possible, we may simply write bT a.s. −→ b. A sequence bT converges almost surely if the probability of obtaining a realiza￾tion of the sequence {Zt} for which convergence to b occurs is unity. Equivalently, 3