3 Stochastic Control References on stochastic control and probability theory include 38,[17, [10, 22,24 2],⑨,39,] 3.1 Some Probability Theory While probability theory, especially quantum probability, will be covered by other speakers at this workshop, we present in this subsection some basic definitions and ideas 3.1.1 Basic Definitions Classical probability theory considers events F, subsets of a sample space and assigns a numerical value 0< P(F)< I to each event F indicating the probability of occurrence of F. The collection of all allowed events is denoted The basic construct of classical probability is the triple(Q, F, P), called a classical probability space To facilitate an adequate framework for integration(expectation), convergence, etc, ere are a number of technical requirements placed on probability spaces. While the set Q of outcomes can be arbitrary (e. g. colors of balls in an urn, the set of real numbers, etc) the collection of allowed events F is required to be a o-algebra. a o-algebra F contains the empty set(∈), is closed under complements(F∈ F implies F={u∈9:wg FI E F), and is closed under countable unions (Filo C F implies Ua FiE F).A pair(@, F)is called a measurable space(on which one or more probability measures may be defined) A probability measure is a function P: F-0, 1] such that (i)0< P(F)< l for all FEF,(ii) P(2)=l, and (iii) if F1, F2,... is a disjoint sequence of events in F, then P(U=1F)=∑1P(F) In many cases, the set of outcomes Q2 will be a topological space, i. e. a set Q2 together with a collection T of subsets(the open sets), called a topology. A topology T contains the empty set, and is closed under arbitrary unions and intersections. If Q2 is discrete then the set of all subsets defines the standard topology. If Q2=R or C(real or complex numbers), the standard topology can be defined by considering all open intervals or discs (and their arbitrary unions and intersections). Given a topological space(Q2, T), the Borel g-algebra B(Q)is the a-algebra generated by the open sets(it is the smallest a-algebra containing all open sets). Events in a Borel o-algebra are called Borel sets. Often, when the topology or o-algebra is clear, explicit mention of them is dropped from the notation Let(1, Fi) and( Q2, F2) be two measurable spaces. A random variable or measurable function is a function X:91→92 such that X-(F)={∈91:X(u)∈F2}∈万1 for all F2 E F2. In particular, a real-valued random variable X defined on(Q, F)is a function X:g→ R such that X-1(B)={∈9:X(u)∈B}∈ for any Borel set BCR. Similarly, we can consider complex-valued random variables. If P is a probability measure on(Q, F), the probability distribution induced by X is Px(B=P(X(B),3 Stochastic Control References on stochastic control and probability theory include [38], [17], [10], [22], [24], [2], [9], [39], [1]. 3.1 Some Probability Theory While probability theory, especially quantum probability, will be covered by other speakers at this workshop, we present in this subsection some basic definitions and ideas. 3.1.1 Basic Definitions Classical probability theory considers events F, subsets of a sample space Ω, and assigns a numerical value 0 ≤ P(F) ≤ 1 to each event F indicating the probability of occurrence of F. The collection of all allowed events is denoted F. The basic construct of classical probability is the triple (Ω, F, P), called a classical probability space. To facilitate an adequate framework for integration (expectation), convergence, etc, there are a number of technical requirements placed on probability spaces. While the set Ω of outcomes can be arbitrary (e.g. colors of balls in an urn, the set of real numbers, etc), the collection of allowed events F is required to be a σ-algebra. A σ-algebra F contains the empty set (∅ ∈ F), is closed under complements (F ∈ F implies F c = {ω ∈ Ω : ω 6∈ F} ∈ F), and is closed under countable unions ({Fi} ∞ i=1 ⊂ F implies S∞ i=1 Fi ∈ F). A pair (Ω, F) is called a measurable space (on which one or more probability measures may be defined). A probability measure is a function P : F → [0, 1] such that (i) 0 ≤ P(F) ≤ 1 for all F ∈ F, (ii) P(Ω) = 1, and (iii) if F1, F2, . . . is a disjoint sequence of events in F, then P( S∞ i=1 Fi) = P∞ i=1 P(Fi). In many cases, the set of outcomes Ω will be a topological space, i.e. a set Ω together with a collection τ of subsets (the open sets), called a topology. A topology τ contains the empty set, and is closed under arbitrary unions and intersections. If Ω is discrete, then the set of all subsets defines the standard topology. If Ω = R or C (real or complex numbers), the standard topology can be defined by considering all open intervals or discs (and their arbitrary unions and intersections). Given a topological space (Ω, τ ), the Borel σ-algebra B(Ω) is the σ-algebra generated by the open sets (it is the smallest σ-algebra containing all open sets). Events in a Borel σ-algebra are called Borel sets. Often, when the topology or σ-algebra is clear, explicit mention of them is dropped from the notation. Let (Ω1, F1) and (Ω2, F2) be two measurable spaces. A random variable or measurable function is a function X : Ω1 → Ω2 such that X−1 (F2) = {ω1 ∈ Ω1 : X(ω1) ∈ F2} ∈ F1 for all F2 ∈ F2. In particular, a real-valued random variable X defined on (Ω, F) is a function X : Ω → R such that X−1 (B) = {ω ∈ Ω : X(ω) ∈ B} ∈ F for any Borel set B ⊂ R. Similarly, we can consider complex-valued random variables. If P is a probability measure on (Ω, F), the probability distribution induced by X is PX(B) = P(X −1 (B)), 22