To show the theorem for an arbitrary Z E C(Q, o(Xt+1), P), use the Monotone Convergence Theorem. L We now use the probability transition function to define a time homogeneous Markov Definition. Let(Q, F, P, )be a filtered probability space and let X be a(P, e Markov process. Let(Qt)t be its probability transition functions. If there is a Q such that Qt=Q for all t= 1, 2, . then X is called a time homogeneous Markov process Proposition. Let(@, F, P, e be a filtered probability space and let X be a time homogeneous(P, F)-Markov process. For any nonnegative integers k, t, let Yt+k C(Q, o(Xt+k), P). Then for each k=0, 1, .. there is a Borel function gk: R-R such that. for each t=0.1 EY++Ft= gk(XL) (19) In particular, there is a Borel function h such that, for each t=0, 1 E[t+1|F]=h(X) 3. 1 Finitestate markov chains in discrete time This is perhaps the simplest class of Markov processes. Let(Q2, F, P)be a probability space and let={x1,x2,……,xn} be a finite set.X:Z+→ a be a stochastic process. Denote by u the vector of probabilities that Xt= i and suppose there isTo show the theorem for an arbitrary Z ∈ L1 (Ω, σ (Xt+1), P), use the Monotone Convergence Theorem. We now use the probability transition function to define a time homogeneous Markov process. Definition. Let (Ω, F, P, F) be a filtered probability space and let X be a (P, F)- Markov process. Let hQti ∞ t=1 be its probability transition functions. If there is a Q such that Qt = Q for all t = 1, 2, ... then X is called a time homogeneous Markov process. Proposition. Let (Ω, F, P, F) be a filtered probability space and let X be a time homogeneous (P, F)-Markov process. For any nonnegative integers k, t, let Yt+k ∈ L 1 (Ω, σ (Xt+k), P). Then for each k = 0, 1, ... there is a Borel function gk : R → R such that, for each t = 0, 1, ... E [Yt+k|Ft ] = gk (Xt). (19) In particular, there is a Borel function h such that, for each t = 0, 1, ... E [Yt+1|Ft ] = h (Xt). (20) 3.1 Finite–state Markov chains in discrete time This is perhaps the simplest class of Markov processes. Let (Ω, F, P) be a probability space and let X = {x1, x2, . . . , xn} be a finite set. X : Z+ → X be a stochastic process. Denote by µt the vector of probabilities that Xt = xi and suppose there is 15