where by rn(i, j)we mean the(i, j) element of the matrix Tn Definition, A set of states is said to be closed if no state outside it can be reached from any state in it Definition. A set of states is said to be ergodic if it is closed and no proper subset is closed Definition. A Markov chain is called irreducible if its only d set is the set of all states Theorem. Let X be a finite state stationary Markov chain with transition matrix T and suppose X irreducible and aperiodic. Then has a unique solution u* and this u* has the property that lim Tu=μ for all po such that u. 1=1 Proof. See Cinlar(1975).E 3.2 Finite-state markov chains in continuous time Let( F, P) be a probability space and let x=a1, 12,..., In be a finite set be a stochastic process. Denote by u(t)the vector of probabilitieswhere by Γn (i, j) we mean the (i, j) element of the matrix Γn . Definition. A set of states is said to be closed if no state outside it can be reached from any state in it. Definition. A set of states is said to be ergodic if it is closed and no proper subset is closed. Definition. A Markov chain is called irreducible if its only closed set is the set of all states. Theorem. Let X be a finite state stationary Markov chain with transition matrix Γ and suppose X irreducible and aperiodic. Then    µ = Γµ µ · 1 = 1 has a unique solution µ ∗ and this µ ∗ has the property that lim t→∞ Γ tµ0 = µ ∗ for all µ0 such that µ · 1 = 1. Proof. See Cinlar (1975). 3.2 Finite–state Markov chains in continuous time Let (Ω, F, P) be a probability space and let X = {x1, x2, . . . , xn} be a finite set. Let X : R+ → X be a stochastic process. Denote by µ(t) the vector of probabilities 17