ENGG40Proatatisticsfoner Chapter 5:Limit Theorems Instructor:Shengyu Zhang
Instructor: Shengyu Zhang
Content Markov and Chebyshev Inequalities The Weak Law of Large Numbers ■ Convergence in Probability The Central Limit Theorem The Strong Law of Large Numbers
Content Markov and Chebyshev Inequalities The Weak Law of Large Numbers Convergence in Probability The Central Limit Theorem The Strong Law of Large Numbers
Background -We will discuss fundamental issues related to the asymptotic behavior of sequences of random variables. Our principal context involves a sequence X1,X2,...of independent identically distributed (i.i.d.)random variables with mean w and variance o2
Background We will discuss fundamental issues related to the asymptotic behavior of sequences of random variables. Our principal context involves a sequence 𝑋1, 𝑋2, … of independent identically distributed (i.i.d.) random variables with mean 𝜇 and variance 𝜎 2
Background Let Sn=X1+…+Xn be the sum of the first n of them. Limit theorems are mostly concerned with the properties of S and related random variables as n becomes very large
Background Let 𝑆𝑛 = 𝑋1 + ⋯ + 𝑋𝑛 be the sum of the first 𝑛 of them. Limit theorems are mostly concerned with the properties of 𝑆𝑛 and related random variables as 𝑛 becomes very large
Background Because of independence,we have var(Sn)=var(X1)+..+var(Xn)=no2 The distribution of s spreads out as n increases Thus S cannot have a meaningful limit. But the situation is different if we consider the sample mean Mn= X1+..+Xn Sn n n
Background Because of independence, we have var 𝑆𝑛 = var 𝑋1 + ⋯ + var(𝑋𝑛) = 𝑛𝜎 2 The distribution of 𝑆𝑛 spreads out as 𝑛 increases Thus 𝑆𝑛 cannot have a meaningful limit. But the situation is different if we consider the sample mean 𝑀𝑛 = 𝑋1+⋯+𝑋𝑛 𝑛 = 𝑆𝑛 𝑛
Background A quick calculation yields, E[Mn]=u,var(Mn)= 02 The variance of M decreases to zero as n increases. Thus the bulk of the distribution of M must be very close to the mean u. This phenomenon is the subject of certain laws of large numbers
Background A quick calculation yields, 𝐄[𝑀𝑛] = 𝜇, var(𝑀𝑛) = 𝜎 2 𝑛 . The variance of 𝑀𝑛 decreases to zero as 𝑛 increases. Thus the bulk of the distribution of 𝑀𝑛 must be very close to the mean 𝜇. This phenomenon is the subject of certain laws of large numbers
Background The laws generally assert that the sample mean Mn converges to the true mean u. These laws provide a mathematical basis for the loose interpretation of an expectation E X=u... .. as the average of a large number of independent samples drawn from the distribution of X
Background The laws generally assert that the sample mean 𝑀𝑛 converges to the true mean 𝜇. These laws provide a mathematical basis for the loose interpretation of an expectation 𝐄[𝑋] = 𝜇 … … as the average of a large number of independent samples drawn from the distribution of 𝑋
Background We will also consider a quantity which is intermediate between Sn and Mn. Zm is defined as follows. 1. subtract nu from S,to obtain the zero-mean random variable Sm-nu 2.then divide by ovn,to form the random variable Sn -nu In= 0√m
Background We will also consider a quantity which is intermediate between 𝑆𝑛 and 𝑀𝑛. 𝑍𝑛 is defined as follows. 1. subtract 𝑛𝜇 from 𝑆𝑛, to obtain the zero-mean random variable 𝑆𝑛 − 𝑛𝜇 2. then divide by 𝜎 𝑛, to form the random variable 𝑍𝑛 = 𝑆𝑛 − 𝑛𝜇 𝜎 𝑛
Background -It can be seen that E[Zn]=0, var[Zn]=1 ■ Since the mean/variance of Zn remain unchanged as n increases,its distribution neither spreads,nor shrinks to a point The central limit theorem is concerned with 口 the asymptotic shape of the distribution of Zm and asserts that Z,becomes the standard normal distribution
Background It can be seen that 𝐄 𝑍𝑛 = 0, var 𝑍𝑛 = 1 Since the mean/variance of 𝑍𝑛 remain unchanged as 𝑛 increases, its distribution neither spreads, nor shrinks to a point. The central limit theorem is concerned with the asymptotic shape of the distribution of 𝑍𝑛 and asserts that 𝑍𝑛 becomes the standard normal distribution
Application Limit theorems are useful for several reasons: ■ (a)Conceptually.They provided an interpretation of expectations/probabilities in terms of a long sequence of identical independent experiments
Application Limit theorems are useful for several reasons: (a) Conceptually. They provided an interpretation of expectations/probabilities in terms of a long sequence of identical independent experiments