Math Review. ECON 510 Chapter 3 Dynamic Optimization 1. Discrete-Time stochastic model Good references: Sydsaeter et al (2005, Chapter 12), Stokey-Lucas(1989, p. 239-259) Sargent(1987, Chapter 1) 1.1. Markov Process What is a Markov process? At t, we know o,.. It and some knowledge of t+1 Lt,t-1 We call ateo a random process. If the distribution function is time-independent r all t, we call ateso a stationary process. If the dependence on past history has a fixed length of time, i. e, there exists an integer n such that r2+1~F({x,…,x-n+1) for all t, we call ateso an nth-order Markov process For example, a first-order Markov process atto is defined by ct, for all tChapter 3 Dynamic Optimization Math Review, ECON 510 1. Discrete-Time Stochastic Model Good references: Sydsaeter et al (2005, Chapter 12), Stokey—Lucas (1989, p.239—259), Sargent (1987, Chapter 1). 1.1. Markov Process What is a Markov process? At t, we know x0,...,xt and some knowledge of xt+1 : xt+1 ∼ Ft(·|xt, xt−1,...,x0). We call {xt}∞ t=0 a random process. If the distribution function is time-independent, i.e., xt+1 ∼ F(·|xt, xt−1,...,x0), for all t, we call {xt}∞ t=0 a stationary process. If the dependence on past history has a fixed length of time, i.e., there exists an integer n such that xt+1 ∼ F(·|xt,...,xt−n+1), for all t, we call {xt}∞ t=0 an nth-order Markov process. For example, a first-order Markov process {xt}∞ t=0 is defined by xt+1 ∼ F(·|xt), for all t. 3—1