EXPECTATIONS AND CONDITIONAL EXPECTATIONS Expectations and Conditional expectations Definition 1 Discrete random variable A random uariable is discrete if the set of outco mes is either finite in number or co untably For a discrete random variable f()=Prob(x Example 1 Bino mial Distribution P(X=m,p=(x)2(1-m)-;x=0,1,2,,n;0≤P≤ Definition 2 Continuous Random Variable The random uariable is contino us f the set of out co mes is ir finitely divisible and, hence not countable For a continuous random variable, b)=/f(x)d≥0 Example 2 Normal Distribution f(=lu 0 ioms of probability require that 1.0≤PobX=x) Definition 3 Expectation of a Random variable The mean, or expected value, of a random wariable is Elz ∑axf(x) f a is discrete f af(a) dr f a is continuous Remark 1 Linearity af expectation a+bx=a+ bela Remark 2 2]≠(E
EXPECTATIONS AND CONDITIONAL EXPECTATIONS 1 Expectations and Conditional Expectations Definition 1 Discrete Random Variable A random variable is discrete if the set of outcomes is either finite in number or countably infinite. For a discrete random variable, f (x) = Prob(X = x). Example 1 Binomial Distribution P (X = x|n, p) = n x p x (1 − p) n−x ; x = 0, 1, 2, ..., n; 0 ≤ p ≤ 1 Definition 2 Continuous Random Variable The random variable is continuous if the set of outcomes is infinitely divisible and, hence, not countable. For a continuous random variable, Prob (a ≤ x ≤ b) = b a f (x) dx ≥ 0. Example 2 Normal Distribution f x|µ, σ2 = 1 √ 2πσ exp − 1 2 x − µ σ 2 , − ∞ 0 Axioms of probability require that 1. 0 ≤Prob(X = x) ≤ 1. 2. x f (x) = 1. Definition 3 Expectation of a Random Variable The mean, or expected value, of a random variable is E [x] = x xf (x) if x is discrete x xf (x) dx if x is continuous Remark 1 Linearity of expectation E [a + bx] = a + bE [x] Remark 2 E x 2 = (E [x])2 and E x 2 − (E [x])2 = σ 2
EXPECTATIONS AND CONDITIONAL EXPECTATIONS Example 3 (0,1) E[1+5]=1+5E[x]=1+5×0=1 E1+5x2=1+5Ex2=1+5×1=6 Definition 4 For tuo random variables, x and y, we say that the conditional distribution of y given a is r(0)=(x0 Remark 3 If a and y are independent. f(-x)=fy(3) Definition 5 A conditional mean(or conditional expectation) is the mean of the conde tio nal distribution and is defined by Ella h uf(ya)dy if y is conti yf(ylr) f y is discrete Note that E E if a and y are independent Example 4 N(0,1) E and r are independent EIy=E[bz + ea E[bx=]+ellul ince e and r are independent, E[E==E[=0, E (Reference: See Greene Appendix B, P. 845-865)
EXPECTATIONS AND CONDITIONAL EXPECTATIONS 2 Example 3 x ∼ N (0, 1) E [1 + 5x] = 1 + 5E [x] = 1 + 5 × 0 = 1 E 1 + 5x 2 = 1 + 5E x 2 = 1 + 5 × 1 = 6 Definition 4 For two random variables, x and y, we say that the conditional distribution of y given x is f (y|x) = f (x, y) fx (x) Remark 3 If x and y are independent, f (y|x) = fy (y) Definition 5 A conditional mean (or conditional expectation) is the mean of the conditional distribution and is defined by E [y|x] = y yf (y|x) dy if y is continuous y yf (y|x) if y is discrete Note that E [y|x] = E [y] if x and y are independent. Example 4 y = bx + ε ε ∼ N (0, 1) ε and x are independent. E [y|x] = E [bx + ε|x] = E [bx|x] + E [ε|x] Since ε and x are independent, E [ε|x] = E [ε] = 0, E [y|x] = bx (Reference: See Greene Appendix B, P. 845—865)
MATRIX ALGEBRA Matrix Algebra 4 pi ar cs i mi riy fif nkp bxrsR A=[NA]=[Alk NNM Ii=laai k= li((K whxrxi txnfiaxs ahx rfiw nkp bxnint h txnfiaxs ahx Ciikp ml xFafir 4 pi ar wah finy fix EEkp m Example 5 N Rfw lxPafir 4 piard wah fimy fix rfw Example 6 Sypp xandEr iar5(Nk= Ni Example 7 2 A=21 321 Da gini3pia(Nk=0I≠k) Example 8 100 020 SFiSnpiar(Nk=OI Ni=NkIifh Example 9 300 It xmaay piaG5 (N:=OI Ni= Nk=1I ith
MATRIX ALGEBRA 3 Matrix Algebra A matrix is an array of numbers, A = [aik] = [A] ik = a11 a12 · · · a1K a21 a22 · · · a2K · · · an1 an2 · · · anK , i = 1, ..., n, k = 1, ..., K where i denotes the row number and k denotes the column number. Column vector: A matrix with only one column. Example 5 vi = a1 a2 . . . an Row vector: A matrix with only one row Example 6 vk = a1 a2 · · · aK Symmetric matrix (aik = aki) Example 7 A = 1 2 3 2 1 2 3 2 1 Diagonal matrix (aik = 0, i = k) Example 8 A = 1 0 0 0 2 0 0 0 3 Scalar matrix (aik = 0, aii = akk, i = k) Example 9 A = 3 0 0 0 3 0 0 0 3 Identity matrix (aik = 0, aii = akk = 1, i = k)
MATRIX ALGI BRA a ndo 2 10 100 E 010 001 Upper triangular matrix(Ek=0o 53 f) a ndo li 11 23 E=012 00 Lower triangular matrix(Ek=Oo 5b f) a ndo mle lb E 210 Matrix Manipulations Tudc, mt, rert c For two matrices, E and PoP is the transpose of E implies that xk= E a ndo mli 1l 123 456 n devrn Arr rert c dcr puFeuduert c For two matrices, E and Pomatrix addition and subtract ion are defined by E+P Er+ E-P=ER a ndo li 1h E 6 133 557° E-P 355
MATRIX ALGEBRA 4 Example 10 A = 1 0 0 0 1 0 0 0 1 Upper triangular matrix (aik = 0, i > k) Example 11 A = 1 2 3 0 1 2 0 0 1 Lower triangular matrix (aik = 0, i < k) Example 12 A = 1 0 0 2 1 0 3 2 1 Matrix Manupulations Transposition For two matrices, A and B, B is the transpose of A implies that bik = aki Example 13 A = 1 2 3 4 5 6 , B = A′ = 1 4 2 5 3 6 Matrix Addition and Subtraction For two matrices, A and B, matrix addition and subtraction are defined by A + B = [aik + bik] A − B = [aik − bik] Example 14 A = 1 2 3 4 5 6 , B = 0 1 0 1 0 1 A + B = 1 3 3 5 5 7 , A − B = 1 1 3 3 5 5
MATRIX ALG BRA Ar r FP5 Fi: g8d -Di d 5 Fi: Gsd6 i sc ds asIf mo 1 mdscl C roe TAdOntc toone yosef ad Nc esd yosef ac t1 T=TC=a1b1+a2b2+.+ anbn t n msx CT= TC=1 t dAe( Gldanrlast dai r t n rlf sf 01 E P 2×1+ 3×01 1+2×0 EP=4×0+5×1+6X04×1+5×0+6×1 0×1+1×40×2+1×50×3+1×6 PE=1×1+0×41×2+0×51×3+0×6 In general EPf PE h Ils atral n ITw(EP)A=E(Pa) Daldctbf dlnlrwTE(P+A)=EP+EA Tcrolysln sfir yosef adr(EP)=Pe Tcrolys In sfiro mudroe ne yosef adr(EPA)=A'P'E SGn i) Vt IGF Dno s dn i r l nads dOrd aso drool r asif mo sfi so nITTOTo A O TIF Sp 1O=1×1+1×2+1×3=6
MATRIX ALGEBRA 5 Inner Product (Dot Product) For two column vectors a and b, their inner product (or dot product) is a ′b = b ′a = a1b1 + a2b2 + · · · + anbn Example 15 a = 1 2 3 , b = 3 2 1 a ′b = b ′a = 1 × 3 + 2 × 2 + 3 × 1 = 10 Matrix Multiplication Example 16 A = 1 2 3 4 5 6 2×3 , B = 0 1 1 0 0 1 3×2 AB = 1 × 0 + 2 × 1 + 3 × 0 1 × 1 + 2 × 0 + 3 × 1 4 × 0 + 5 × 1 + 6 × 0 4 × 1 + 5 × 0 + 6 × 1 = 2 4 5 10 2×2 BA = 0 × 1 + 1 × 4 0 × 2 + 1 × 5 0 × 3 + 1 × 6 1 × 1 + 0 × 4 1 × 2 + 0 × 5 1 × 3 + 0 × 6 0 × 1 + 1 × 4 0 × 2 + 1 × 5 0 × 3 + 1 × 6 = 4 5 6 1 2 3 4 5 6 3×3 In general AB = BA. • Associative law: (AB) C = A (BC) • Distributive law: A (B + C) = AB + AC • Transpose of a product: (AB) ′ = B′A′ • Transpose of an extended product: (ABC) ′ = C′B′A′ Sum of Values Denote i a vector that contains a column of ones. Then, n i=1 xi = x1 + · · · xn = i ′x Example 17 x = 1 2 3 , i = 1 1 1 , i ′x = 1 × 1 + 1 × 2 + 1 × 3 = 6
+01520]6(350 puo t-pwuc Ths Om P qul SOFk Ds SISmSLIOIL T NumL vSNFCs 10 ∑ a ndo m2 l s=1Is's=1×1MI×IMB×B=14 B puo t-I ot uue Ths (m Fk thS KCFAMOFk R p mhlumL vMFCos ILAE 1o ∑Xm=sE=Es (RSASCSLNS SSS(SSLS AKKSLAH A, P. 803-845
MATRIX ALGEBRA 6 Sum of Squares The sum of squares of the elements in a column vector x is n i=1 x 2 i = x ′x Example 18 x = 1 2 3 , x ′x = 1 × 1 + 2 × 2 + 3 × 3 = 14 Sum of Products The sum of the products of two column vectors x and y is n i=1 xiyi = x ′y = y ′x (Reference: See Greene Appendix A, P. 803—845)