1907年由俄罗斯数学家马尔可夫（A.Markov）提出， 并由蒙特-卡罗(Mote-Carlo)加以发展。 用于分析随机事件未来发展变化的趋势，即利用某 一变量的现状和动向去预测该变量未来的状态及动 向，以预测未来某特定时期可能发生的变化，以便 采取相应的对策

（一） 几种重要的纯不连续马氏过程 （1） Poission 过程（专门讲解） （2） 纯增殖过程（人口问题）

参数连续状态离散的马氏过程的转移概率 定义:设X={X(1),t≥0}是取值于状态空间S的随机过程,S是 有限或无限可列的,如果对于任意的正整数n,任意的

（一） 到达与相通 定义：对给定的两个状态

Specific common processes White-noise: Autocorrelation is Dirac-delta function; PSD is flat; integral of power under PSD is variance of process(true in general) First-order Gauss-Markov process (one of most models common in Kalman filtering)

一、状态停留时间 二、状态微分方程 三、生灭过程

记忆特性随机过程 一、纯粹独立随机过程 二、独立增量过程 三、Markov过程

Leslie Pack Kaelbling, Michael. Littman and Anthony R. Cassandra, \Planning and Acting in Partially Observable Stochastic Domains,\ Artificial Intelligence, Vol. 101, 1998. j. Pineau,g. Gordon&s. Thrun\Point-based- value iteration: An anytime algorithm for POMDPs\. International Joint Conference on Artificial Intelligence(JCAI) Acapulco, Mexico. Aug

How Might a mouse search a Maze for Cheese? heese · State Space Search? As a Constraint Satisfaction Problem? Goal-directed Planning As a rule or production System? What is missing? Ideas in this lecture Objective is to accumulate rewards rather than goal states Task is to generate policies for how to act in all situations rather than a plan for a single starting situation

Consistency e Under the Gauss-Markov assumptionS OLS IS BLUE, but in other cases it wont always be possible to find unbiased estimators o In those cases, we may settle for estimators that are consistent, meaning as n→>∞,the distribution of the estimator collapses to the parameter value Economics 20- Prof anderson