3.1. 3.7 (1) 100 因为有限状态的马氏链平稳分布一定存在 ()=x()P=000)v302/3|=032/3) 兀=TP 01/32/3 解方程组得 x(2)=x(1)P=(192/92/3) 兀=(21/62,23/62,9/31) E(X2)=1×1/9+2×2/9+3×2/3=23/9 21/6223/629/31 imn,P"=2J6223/62931 π(1)=(01/32/3) l6223/62 E(x2X1=1)没有意义 当(0)=(O)P时,即x(0)=(21/62,23/62,9/31)时,马氏链 E(XxXx1=2)=∑P2=1×13+3×2/3=73 是平稳序列 (n)=x(0)=(21/62,23/62,9/31) E(xx1=3)=∑P1=2×134+3×2/3=8/3 ∴EXn=1×2162+2×23/62+3×9/31=121/6 EX2=1×21/62+4×2362+9×9/31=275/62 x(2)=(/92/92/3) DXn=EXn2-(EXn)=24093844 E(xx2=1)=∑P1=1 3.11 根据马氏性,E(Xx2=2)=7/3,E(M3x2=3)=8/3 (1) E(e2m-|x0,x,…,x,) P(T==3)=P1=0 P(T=2Xx=3)=∑P(X2=1X 3) P(X >P(2=1x,=). P(X, =iLo=3) P3P1=(/32/3) P(T=3X0=3) ∑∑P(x1=1,x2=1,X=八|X0=3 N(1-e ∑∑P,PP (2) 定义 (/32/3) 02/3)(v/3 2/27 T=min{:n≥0,Xxn∈{0,N},X0=k}, TN=min(n: n20,X=N,Xo=k) P(T≥4X0=3)=1-1/9-2/27=2227 VN=P(TN<+ooO=k)=P(X,=N) E(TA4X0=3)=2×/9+3×2/27+4×227=10027 可知7和T是关于{Xn,n≥0}的停时,且P3(k)=x 因为{e-2ax",n≥0}是鞅,且 P(T1=1)=0 设S0={2,3},当k≥2时, ①因为两边带有吸收壁的有限状态马氏链的中间状态为瞬时 态,所以P(T<∞)=1 P(TM=k) P(X=1,X∈S0,0<1<k|x0=1) 02/3)(1/3 ③imn,E -2arT (/32/3) 满足停时定理的条件,所以 e(e )=e(e) E(TM) P(X=N=N, P(X=0)=l-l k. 2 十 3.14 (1)用停时定理证明 定义随机变量z=0,Zn=Xn-Xn,n≥1,可知Zn的分布律为
3.1. (1) ( ) ( ) π(2) π(1) ( ) 1 9 2 9 2 3 0 1 3 2 3 0 1 3 2 3 1 3 0 2 3 1 0 0 π(1) π(0) 0 0 1 1 1 = ⋅ = = = ⋅ = ⋅ P ∵ P ( ) 1 1 9 2 2 9 3 2 3 23 9 ∴ E X2 = × + × + × = ∵ π(1) = ( ) 0 1 3 2 3 ( 3) 2 1 3 3 2 3 8 3 ( 2) 1 1 3 3 2 3 7 3 ( 1) 3 1 2 1 3 3 1 2 1 2 2 1 = = ⋅ = × + × = = = ⋅ = × + × = ∴ = ∑ ∑ = = i i i i E X X i p E X X i p E X X 没有意义 ∵ π(2) = ( ) 1 9 2 9 2 3 ( 1) 1 3 1 3 2 ∑ 1 = ∴ = ⋅ = i i E X X ＝ i p 根据马氏性, ( 2) 7 3, ( 3) 8 3 E X3 X2 = = E X3 X2 = = (2) P(T = 1 X0 = 3) = p31 = 0 ( ) 3 0 2 10 2 3 21 10 2 3 3 1 2 ( 2 3) ( 1, 3) ( 1 ) ( 3) 1 3 13 23 19 0 i i i i i PT X P X X iX PX X i PX iX p p = = = = == = = = = = =⋅ = = = ⋅= ⋅ = ∑ ∑ ∑ ( ) 0 3 3 3 21 0 2 2 3 3 3 1 2 2 ( 3 3) ( 1, , 3) 0 23 13 1 3 2 3 2 27 13 23 0 i j j ji i i j PT X PX X iX jX p pp = = = = = = = = == = = ⋅⋅ = ⋅ ⋅= ∑∑ ∑∑ ∴ P(T ≥ 4 X0 = 3) = 1−1 9 − 2 27 = 22 27 ∴ E(T ∧ 4 X0 = 3) = 2×1 9 + 3× 2 27 + 4× 22 27 = 100 27 (3) P(T11 = 1) = 0 设 {2,3} S0 = , 当 k ≥ 2 时, ( ) 11 0 0 2 2 1 ( ) ( 1, ,0 1) 0 23 13 13 23 23 0 13 2 3 k l k k k PT k PX X S l kX − − − = = = ∈ ≤ →∞ > = − →∞ E e I P T n T n n aX n T 满足停时定理的条件, 所以 aX aX ak E e E e e 2 T 2 2 ( ) ( ) − − 0 − = = , P XT = N = VN P XT = = −VN ∵ ( ) , ( 0) 1 aN ak N ak N N aN e e V e V V e 2 2 2 2 1 1 (1 ) − − − − − − ∴ = ∴ ⋅ + − = 3.14 (1) 用停时定理证明: 定义随机变量 0 0, , 1 Z Z X Yn = n nn =− ≥ , 可知 Zn 的分布律为
P(Zn=1)=P(Xxn=1,yn=0)=P1(1-P2) P(T=2k|X6=1) P(Z,=0)=P(X,=0,,=0)+P(X=l, Y =1) P(X2k=3X=2,Xm=12k>为奇数m为偶数>0,X=1 P2+(1-P1)1-P2) P(Zn=-1)=P(Xn=0,Yn=1)=P2(1-n) (P2P21)·P2P23=p P(T=2k+1|X=1) 设M6=0,Mn=∑2,n21,易知{M,n≥0是马氏链 P(x2=0,X=2,Xm=12k>D为奇数m为偶数>0x0=1) 定义 =(Pi2 P2). Po=p'q NM=min{n:n≥0,Mn=-M P(T=k|X6=1)=P k为奇数;+Pq(为偶数 可知N和NM是{zn,n≥0}的停时.观察所证结果,想到验证 P(xn=3{X6=1)=∑P(T=2k|X0=1) 其中 P 关于{Zn,n≥0}是鞅,如下 p∑pq (n|0,…zn p2/(-pg) E(Mnm|0,…,Zn)=2,E(x2) ·(An(1-P2)+P1P2+(1-p1(1-P2)+AP2(1-P1)) 3.20. 并且 ①可以将M和-M看成吸收态,所以P(Nn)=0 丌=TP 满足停时定理的条件,所以 解方程组得 E(x)=E(x-)=1 丌=(44/81,1/3.10/81) 又因为非周期不可约链的极限分布就是它的平稳分布, E(2)=P(Mr=-M)+-(1-P(Mr=-M) 1-元 P(M M imE(xX0=1)=1×4181+2×/3+3×10/81=128/81 PM=-M)即为误判概率,得证 T1 n:n>0,Xn∈S0} min{n:n>0,x=1l≤i≤n-1Xn∈S} ①方法1,用Wald等式 因为{znn21}独立同分布,EZn=P1-P2T,Xn=l; {zn,n≥1}是停时,且N符合PH分布,EN=a(l-P)T,X=11≤is71-1X,∈S,7≤j≤n-1,Xn=l 其中P为瞬时态集的转移矩阵根据书中P52中的Wad等式,因此m-m和U1-均只是X,…,X的函数即了和r都是关 于{Xn,n≥0}的停时 E∑Z)=(EZEN) 通过观察状态图,可以得到 P(T=k EZ.=P,-P2 P(x1=11sl≤k-1X∈S0|X6=1) E(∑Zn)=E(M) =n1(p12+p1)=(58)-(/4+1/8) 1+4)M、1 M·(1-_1 35-/84 (P1-P2)M+1) 由上面(2)可知r1=T+r',其中r1'为PH分布,其对应的状 方法2,用PH分布(略) 态空间为S=SS0',S"={2,3}为瞬时态集,S={}为吸收态, 3.18 步转移概率矩阵为 观察状态图,可以知道X0=1时,首次到达Xn=0需要经过 1/21/61/3 奇数步,而首次到达Xn=3需要经过偶数步,所以需要分奇偶 P 3/ 可以把上面的一步转移概率矩阵看成
1 2 12 1 2 2 1 ( 1) ( 1, 0) (1 ) ( 0) ( 0, 0) ( 1, 1) (1 )(1 ) ( 1) ( 0, 1) (1 ) n nn n n n nn n nn PZ PX Y p p PZ PX Y PX Y pp p p PZ PX Y p p == = = = − == = =+ = = = +− − =− = = = = − 设 0 1 0, , 1 n n k k M M Zn = == ≥ ∑ , 易知{ , 0} M n n ≥ 是马氏链. 定义 min{ : 0, } N nn M M −M n = ≥ =− , 可知 N N 和 是 −M { , 0} Z n n ≥ 的停时. 观察所证结果, 想到验证 { } 1 2 2 1 (1 ) , 0 , (1 ) Mn n p p V n p p λ λ − − =≥ = − 其中 关于{ , 0} Z n n ≥ 是鞅, 如下: 1 1 1 0 0 1 1 2 12 1 2 2 1 ( ,, ) ( ,, ) ( ) ( (1 ) (1 )(1 ) (1 )) nn n n n n n n MZ M Z n M M n EV Z Z E ZZ E p p pp p p p p V λλ λ λ λλ λ λ + + + −− − − − − − =⋅ = = ⋅ − + +− − + − = = " " 并且: ①可以将 M和 看成吸收态 − M , 所以 P N( )1 →∞ ⋅ ≤ ⋅ >= 满足停时定理的条件, 所以 0 ( ) ( )1 MT M E E λ λ − − = = ( ) ( )+ (1 ( )) 1 1 ( ) 1 M N M M T T M N MM M E PM M PM M PM M λλ λ λ λλ λ − − − − = =− − =− − ∴ =− = = − + ∵ ( ) PM M T = − 即为误判概率, 得证. (2) ① 方法 1, 用 Wald 等式 因为{ , 1} Z n n ≥ 独立同分布, EZ p p n = 1 2 − > = = ⋅ ⋅⋅ = 为奇数 为偶数 ( ) 1 0 2 0 1 1 12 21 10 ( 2 1 1) ( 0, 2, 1,2 , 0 1) klm k k k PT k X PX X X k l m X p p p pq − − =+ = = = = => > = = ⋅ ⋅= 为奇数 为偶数 ( 1) / 2 ( 1) / 2 / 2 1 / 2 1 1 0 {} {} ( 1) k k kk PT k X p q I p q I k k − + +− ∴ = == + 为奇数 为偶数 1 0 10 1 2 11 1 2 ( 3 1) ( 2 1) (1 ) T k k k k P X X PT k X p pq p pq ∞ = ∞ − − = = == = = = = − ∑ ∑ 3.20. (1) 58 14 18 13 12 16 34 14 0 P = 因为有限状态的马氏链平稳分布一定存在, ∴π = πP , 解方程组得 π = (44 81,1 3,10 81) 又因为非周期不可约链的极限分布就是它的平稳分布, j n n ij lim p π ( ) ∴ →∞ = lim ( 1) 1 44 81 2 1 3 3 10 81 128 81 ∴ 0 = = × + × + × = →∞ E Xn X n (2) min{ : , 1,1 1, , 1, 1} min{ : , 1} min{ : 0, 1,1 1, } min{ : 0, } 1 1 0 1 1 1 0 1 0 = > = ≤ ≤ − ∈ ≤ ≤ − = = > = = > = ≤ ≤ − ∈ = > ∈ i j n n i n n n n T X i T X S T j n X n n T X n n X i n X S T n n X S τ 因此 { } { } T1 n 1 n I I = 和 均只是 τ = X Xn , , 1 " 的函数, 即 1 1 T 和τ 都是关 于{X ,n ≥ 0} n 的停时. (3) 通过观察状态图, 可以得到: ( ) ( ) 1 0 0 1 1 11 12 13 1 ( ) ( 1,1 1, 1) ( ) 58 14 18 35 8 l k k k k k PT k PX l k X S X p pp − − − = = = ≤≤ − ∈ = = ⋅+ = ⋅+ = ⋅ 5 8 8 3 ( ) 3 5 8 1 1 1 1 1 1 = = ∴ = ⋅ ⋅ ∑ ∑ ∞ = − − ∞ = − k k k k k k E T k 由上面(2)可知 ' 1 1 1 τ = T +τ , 其中 ' 1 τ 为 PH 分布, 其对应的状 态空间为 ' ' 0 S = S ∪S , } S'= {2,3 为瞬时态集, } ' {1 S0 = 为吸收态, 一步转移概率矩阵为 = 1 4 1 8 5 8 1 4 0 3 4 1 2 1 6 1 3 ~ P 可以把上面的一步转移概率矩阵看成
P=1403/4 01 而不影响对PH分布的计算结果,其中 1/216 l/40 确定这个过程的初始分布x(0)=(ax0,a),由上可知 P(Xn=2)=P2/(P12+p3)=2/3 P(Xn=3)=P13/(p2+p3)=1/3, 所以x(O)=(.,2/3,1/3),其中a=(2/3,1/3) P(T,=k)=aP Po E(x)=∑kaP-B=a(-P)e (2/31/3) =74/33 E(x1)=E(T+x1)=8/3+74/33=5411 P(N(3)=0) P(X1=1,X2=1,X3=1X0=1) (58)2=2 P(N(3)=2) =P(T1=1,r1=2,73=3) P(x3∈S,X2=1,Xx1∈S0|X=1) (P2P13) P21 (P2+P13) 17/256 P(N(3)>2)=0 P(N(3)=1)=1-125/512-17/256=353/512 同理 P(N(4)=2) P(T=1,r1=2,72=3)+P(T1=1,r1=2,T2=4) +P(T1=1,r1=3,72=4)+P(T1=2,r1=3,72=4) 185/1024
= 0 0 1 1 4 0 3 4 1 2 1 6 1 3 ' ~ P , 而不影响对 PH 分布的计算结果, 其中, = = 3 4 1 3 , 1 4 0 1 2 1 6 P P0 . 确定这个过程的初始分布 π(0) = ( ) α 0 ,α , 由上可知 ( 2) ( ) 2 3 P XT1 = = p12 p12 + p13 = , ( 3) ( ) 1 3 P XT1 = = p13 p12 + p13 = , 所以 π(0) = ( ) 0, 2 3,1 3 , 其中α = (2 3,1 3) . 0 1 1 P( ' k) P P k − ∴ τ = =α 1 1 1 0 1 1 ( ') ( ) 12 16 1 (2 3 1 3) 14 1 1 74 33 k k E τ αα kP P I P e ∞ − − = − ∴ = =− − =⋅ ⋅ − = ∑ 1 11 E ET ( ) ( ') 8 3 74 33 54 11 τ τ = +=+ = (4) ( ) 1230 3 ( (3) 0) ( 1, 1, 1 1) 5 8 125 512 P N PX X X X = = ==== = = 11 3 3 02 1 00 21 12 13 12 13 31 ( (3) 2) ( 1, 2, 3) ( , 1, 1) ( )( ) 1 3 (1 4 1 8) 3 8 3 4 17 256 P N PT T PX S X X S X p pp pp p τ = = == = = ∈ =∈ = = ⋅ ⋅+ = ⋅⋅ = ( (3) 2) 0 ( (3) 1) 1 125 512 17 256 353 512 P N P N > = = =− − = 同理: 11 2 11 2 11 2 1 1 2 ( (4) 2) ( 1, 2, 3) ( 1, 2, 4) ( 1, 3, 4) ( 2, 3, 4) 185 1024 P N PT T PT T PT T PT T τ τ τ τ = = = = =+ = = = + = = =+ = = = =