Why do I need probability in a deterministic-world?Deterministic systems can be unpredictable in the classical sense and require aprobabilistic description. In this section we elaborate on this observation.In 1963, E.N.Lorenz introduced the following system of ODEs:i=o(y -a),(1)g= z(r-z2) -y,之=ry-bzwhere o, r, b are positive parameters. (1) has become the paradigm of a systemdisplaying chaos, defined as aperiodic long-time behavior in a deterministicsystem which erhibits sensitive dependence on initial conditions. This definitioncontains three main elements:1. Aperiodic long-time behavior, meaning that generic bounded trajectoriesdo not settle on fixed points or periodic orbits.2. Deterministic, meaning that there is no random input in the system.3. Sensitive dependence on initial conditions, meaning that nearby trajec-tories separate exponentially fast in time, i.e. the system has a positiveLyapunov exponent.The standard parameter setting for studying Lorenz system in the chaoticregimeis=10,r=28.b =8/3.Numerical experimentsconducted atthis values confirm the properties above for the solutions of (1). All trajectoriesof (1)eventuallysettlea set of zero volume in phase-space, referred to astheLorenzattractor andshown on figure l.Lorenzattractor hasa characteristicbutterfly shape and the trajectories spiral outward on one wing before makinga transition to the vicinity of the center of the other wing, and repeating theprocess - see figure 2. The sensitive dependence to initial conditions is appar-ent herefrom thefact thattwonearbytrajectories eventually jumpat differenttimes from one wing to the other, as illustrated on figure 2.In the original paper, Lorenz noted the following remarkable fact. Let mnbethevalueof the nth maximum of z.Plotting intheplanethe successivepairs(mn, mn+1) for n = 1, 2,... produces a figure where the points (mn, mn+1) lineup on a graph with a tent shape. Therefore it is as if the solutions of (1) leadto a symbolic dynamics for mn,mn+1=f(mn),with the function f whose graph is shown in figure 3.Generally,given a function f,a recurrence relation like this one defines what is called a one-dimensionalmap. These are much simpler to analyze than ODEs like (1), and we now use1
1 Why do I need probability in a deterministic world? Deterministic systems can be unpredictable in the classical sense and require a probabilistic description. In this section we elaborate on this observation. In 1963, E. N. Lorenz introduced the following system of ODEs: x˙ = σ(y − x), y˙ = x(r − z) − y, z˙ = xy − bz, (1) where σ, r, b are positive parameters. (1) has become the paradigm of a system displaying chaos, defined as aperiodic long-time behavior in a deterministic system which exhibits sensitive dependence on initial conditions. This definition contains three main elements: 1. Aperiodic long-time behavior, meaning that generic bounded trajectories do not settle on fixed points or periodic orbits. 2. Deterministic, meaning that there is no random input in the system. 3. Sensitive dependence on initial conditions, meaning that nearby trajectories separate exponentially fast in time, i.e. the system has a positive Lyapunov exponent. The standard parameter setting for studying Lorenz system in the chaotic regime is σ = 10, r = 28, b = 8/3. Numerical experiments conducted at this values confirm the properties above for the solutions of (1). All trajectories of (1) eventually settle a set of zero volume in phase-space, referred to as the Lorenz attractor and shown on figure 1. Lorenz attractor has a characteristic butterfly shape and the trajectories spiral outward on one wing before making a transition to the vicinity of the center of the other wing, and repeating the process – see figure 2. The sensitive dependence to initial conditions is apparent here from the fact that two nearby trajectories eventually jump at different times from one wing to the other, as illustrated on figure 2. In the original paper, Lorenz noted the following remarkable fact. Let mn be the value of the nth maximum of z. Plotting in the plane the successive pairs (mn, mn+1) for n = 1, 2, . . . produces a figure where the points (mn, mn+1) line up on a graph with a tent shape. Therefore it is as if the solutions of (1) lead to a symbolic dynamics for mn, mn+1 = f(mn), with the function f whose graph is shown in figure 3. Generally, given a function f, a recurrence relation like this one defines what is called a one-dimensional map. These are much simpler to analyze than ODEs like (1), and we now use 1
Figure l: A given trajectory in the (r,y) plane. The trajectory settles on theLorenzattractor.1520Figure 2: The r-component of two trajectories whose initial conditions differ by10-4
−20 −15 −10 −5 0 5 10 15 20 −30 −20 −10 0 10 20 30 Figure 1: A given trajectory in the (x, y) plane. The trajectory settles on the Lorenz attractor. 0 5 10 15 20 25 −20 −15 −10 −5 0 5 10 15 20 Figure 2: The x-component of two trajectories whose initial conditions differ by 10−4 . 2
323453840424446Figure 3: The successive pairs (mn, mn+1) for n = 1, 2,.... Thus the dynamics+1of mn seems reducible to that of a one-dimensional map.a map similar (up to rescaling) to the one for mn where the function f is givenby(2xif r≤1/2f(a) [2- 2r if >1/2If ro [0,1], the recurrence relation(2)En+1=f(rn),then defines a dynamics on the interval [o, 1]- see figure 4. This map is referredto as the tent map and it is not difficult to analyze its properties - in view ofthe analogy with the dynamics for mn, these will help elucidate some of theproperties of Lorenz system.Recall that to every number r[0, 1] one can associateda sequence (ajljeN,with αj = 0 or Qj = 1, such thatCaj2-jT=JENThe sequence (ajljen is called the binary representation of r, and we shallwrite = [α1,α2,.. ]. Now suppose the initial condition ro for the map in (2)isFo =[α1,α2,]If aα1 = 0, then ro < 1/2, and it follows that1=[Q2,03,.]3
30 32 34 36 38 40 42 44 46 48 30 32 34 36 38 40 42 44 46 48 Figure 3: The successive pairs (mn, mn+1) for n = 1, 2, . . . Thus the dynamics of mn seems reducible to that of a one-dimensional map. a map similar (up to rescaling) to the one for mn where the function f is given by f(x) = ( 2x if x ≤ 1/2 2 − 2x if x > 1/2. If x0 ∈ [0, 1], the recurrence relation xn+1 = f(xn), (2) then defines a dynamics on the interval [0, 1] – see figure 4. This map is referred to as the tent map and it is not difficult to analyze its properties – in view of the analogy with the dynamics for mn, these will help elucidate some of the properties of Lorenz system. Recall that to every number x ∈ [0, 1] one can associated a sequence {αj}j∈N, with αj = 0 or αj = 1, such that x = X j∈N αj2 −j . The sequence {αj}j∈N is called the binary representation of x, and we shall write x = [α1, α2, . . .]. Now suppose the initial condition x0 for the map in (2) is x0 = [α1, α2, . . .]. If α1 = 0, then x0 < 1/2, and it follows that x1 = [α2, α3, . . .]. 3
-42XoX30Xi1f (B)f(B)Figure 4: The tent map. The figure shows the first three iterates of ro, and thepre-image of the set B.Similarly if a1 = 1, then ro ≥ 1/2, and it follows that1 = [Nα2, N3,..],where N is the negation operator, defined so that N0 = 1 and N1 = 0. If oneagrees that No = I (identity operator), these two expression can be combinedintoTi = [Na102, Na1a3.. ]Iterating the argument givesTn= [Nai+a2.+anan+1,Nai+a.+anan+2..],(3)which gives an explicit expression for the nth iterate of the map.(3) shows that most trajectories in the tent map are aperiodic. Indeed, onlythe rational numbers in [0,1], i.e. these numbers for whichT=[ai,...ap,Bi....,Ba,βi,...,βa....]for some p,q e N, are such that they eventually settle to the only fixed point = O, or to a periodic orbit (both unstable), and these numbers form a zeromeasure set. All the irrational numbers in [0, 1], one the other hand, lead toaperiodic orbits. (3) also shows that the solution of the tent map exhibits sensitive dependence on initial conditions. Consider indeed two initial conditions,ro and yo such that[Fo-yol ≤2-N.4
B x 2 x 3 x f (B) −1 0 x f (B) −1 0 0 1 1 1 Figure 4: The tent map. The figure shows the first three iterates of x0, and the pre-image of the set B. Similarly if α1 = 1, then x0 ≥ 1/2, and it follows that x1 = [Nα2, Nα3, . . .], where N is the negation operator, defined so that N0 = 1 and N1 = 0. If one agrees that N0 = I (identity operator), these two expression can be combined into x1 = [N α1 α2, Nα1 α3, . . .]. Iterating the argument gives xn = [N α1+α2+···+αn αn+1, Nα1+α2+···+αn αn+2, . . .], (3) which gives an explicit expression for the nth iterate of the map. (3) shows that most trajectories in the tent map are aperiodic. Indeed, only the rational numbers in [0, 1], i.e. these numbers for which x = [α1, . . . , αp, β1, . . . , βq, β1, . . . , βq, . . .] for some p, q ∈ N, are such that they eventually settle to the only fixed point x = 0, or to a periodic orbit (both unstable), and these numbers form a zero measure set. All the irrational numbers in [0, 1], one the other hand, lead to aperiodic orbits. (3) also shows that the solution of the tent map exhibits sensitive dependence on initial conditions. Consider indeed two initial conditions, x0 and y0 such that |x0 − y0| ≤ 2 −N . 4
for some arbitrary N e N. Thus the first N binary digits of ro and yo areidentical, and they may differ only afterwards, i.e. they are very close if N islarge.Yet, it is immediate from (3)that the bound above deteriorates rapidlyin time and, in particular, one can only conclude that[n - yn/ ≤1as soonasn>N.This indicates that, despite the deterministic nature of the tent map, it isillusory to try to make long-term predictions about the solutions because smallerror in the initial conditions areinevitable but eventuallydominate the solution.Errors in initial conditions can be represented probabilistically.Therefore, aprobabilistic description of the tent map seems appropriate, and we focus onthis next.Suppose that the initial condition of the tent map is a random variable withdistribution μo, i.e. for every set B e [0,1],μo(B) = Prob(ro E B),and of course μ([0,1l) = 1. This may be a way to represent errors in initialconditions. For instance if our measurement of the initial condition gives r,but there is a possible error on this measurement, one may take the uniformdistribution on [r 8, + ] for μo.How does μo evolve by the tent map? In other words, if μo is the probabilitydistribution of o, and i = f(ro), what is the distribution μ1 of i? Theanswer is easily found from the definition:μi(B) = Prob[r1 E B) = Prob[f(ro) E B).Denote by f-1(B) the pre-image of B, i.e. the set such that f(f-1(B)) = B seefigure 4.Thenμi(B) = Prob[ro E f-1(B)) = μo(f-1(B)),It is straightforward to iterate the argument. If μn is the the probability distri-bution of rn, the nth iterate of the map, thenμn(B)= μo(f-n(B).Now, what do we gain from this viewpoint? Well, while the evolution of rn isaperiodic and unpredictable, the evolution of μn is simpler. In fact, it is easyto show (do it!) that if μo is continuous, then μn converges to the uniformdistribution on [o,1] as n -→ oo, i.e.(4)μn →μ as n →oo with μ(B) = length(B)(In particular, μ(dr) = dr.) The limiting distribution μ in (4) is called theinuariant measure of themap.It is the unique continuous measurewhich is leftinvariantbythemap, i.eμ(B) = μ(f-1(B).5
for some arbitrary N ∈ N. Thus the first N binary digits of x0 and y0 are identical, and they may differ only afterwards, i.e. they are very close if N is large. Yet, it is immediate from (3) that the bound above deteriorates rapidly in time and, in particular, one can only conclude that |xn − yn| ≤ 1 as soon as n > N. This indicates that, despite the deterministic nature of the tent map, it is illusory to try to make long-term predictions about the solutions because small error in the initial conditions are inevitable but eventually dominate the solution. Errors in initial conditions can be represented probabilistically. Therefore, a probabilistic description of the tent map seems appropriate, and we focus on this next. Suppose that the initial condition of the tent map is a random variable with distribution µ0, i.e. for every set B ∈ [0, 1], µ0(B) = Prob{x0 ∈ B}, and of course µ([0, 1]) = 1. This may be a way to represent errors in initial conditions. For instance if our measurement of the initial condition gives x, but there is a possible error δ on this measurement, one may take the uniform distribution on [x − δ, x + δ] for µ0. How does µ0 evolve by the tent map? In other words, if µ0 is the probability distribution of x0, and x1 = f(x0), what is the distribution µ1 of x1? The answer is easily found from the definition: µ1(B) = Prob{x1 ∈ B} = Prob{f(x0) ∈ B}. Denote by f −1 (B) the pre-image of B, i.e. the set such that f(f −1 (B)) = B – see figure 4. Then µ1(B) = Prob{x0 ∈ f −1 (B)} = µ0(f −1 (B)). It is straightforward to iterate the argument. If µn is the the probability distribution of xn, the nth iterate of the map, then µn(B) = µ0(f −n (B)). Now, what do we gain from this viewpoint? Well, while the evolution of xn is aperiodic and unpredictable, the evolution of µn is simpler. In fact, it is easy to show (do it!) that if µ0 is continuous, then µn converges to the uniform distribution on [0, 1] as n → ∞, i.e. µn → µ as n → ∞ with µ(B) = length(B). (4) (In particular, µ(dx) = dx.) The limiting distribution µ in (4) is called the invariant measure of the map. It is the unique continuous measure which is left invariant by the map, i.e. µ(B) = µ(f −1 (B)). 5
The importance of the invariant measure is made apparent from Birkhoff ergodictheorem, which states that, given any test function on [O,1], one has13Zn(Fj)→/n(r)daasn→oo,noJofor almost all initial conditions ro (ie. except for a set of Lebesgue measurezero).6
The importance of the invariant measure is made apparent from Birkhoff ergodic theorem, which states that, given any test function η on [0, 1], one has 1 n nX−1 j=0 η(xj ) → Z 1 0 η(x)dx as n → ∞, for almost all initial conditions x0 (i.e. except for a set of Lebesgue measure zero). 6