which is the desired second half of(6). This establishes the dynamic programming prin- de Regularity. By regularity we mean the degree of continuity or differentiability; i.e. of smoothness. The regularity of value functions is determined by both the regularity of the data defining it(e.g. f, L, v ), and on the nature of the optimization problem. In many applications, the value function can readily be shown to be continuous, even Lipschitz, but not C in general. The finite horizon value function V(t, r)defined by (5)can be shown to be bounded and Lipschitz continuous under the following (rather strong) assumptions on the problem data: f, L, y are bounded with bounded first order derivatives. We shall It should be noted that in general it can happen that value functions fail to be con tinuous. In fact, the viscosity theory is capable of dealing with semicontinuous or even only locally bounded functions Viscosity solution. Let us re-write the HJ equation(7)as follows: at v(t, 2)+ H(,VV(, 2))=0 in(o, t1)X R with a new definition of the hamiltonian {-入·f(x,)-L(x,t)} The sign convention used in(7)relates to the maximum principle in PDE, and is compat ible with the convention used for the general HJ equation(1). Note that the Hamiltonian is now conver in入. A function V E C(to, til x R)is a viscosity subsolution(resp. supersolution) of (7) if for all∈Ch(to,t1)×Rn) 0 o(s0,x0)+H(xo,Vo(s0,x0)≤0(resp.≥ (54) at every point(so, o) where V-o attains a local maximum(resp. minimum). V viscosity solution if it is both a subsolution and a supersolution We now show that the value function V defined by(5)is a viscosity solution of(7) Subsolution property. Let EC((to, t1)xR)and suppose that V-o attains a local maximum at(so, to); so there exists r >0 such that V(t, r)-p(t, a)<V(so, to)-(so, o)Vlr-aol<r, t-sol <r.(55) Fix u(t)=u E U for all t(constant control) and let 5( denote the corresponding state trajectory with f(so)=co. By standard ODE estimates, we have IE(so+h)-tol<r for all 0 s hs ho(some ho>0)-since U and f are bounded. Then by(55) V(so +h, 5(so+ h))-(so+h, S(so+h))<v(so, to)-o(so, to (57)which is the desired second half of (6). This establishes the dynamic programming prin￾ciple (6). Regularity. By regularity we mean the degree of continuity or differentiability; i.e. of smoothness. The regularity of value functions is determined by both the regularity of the data defining it (e.g. f, L, ψ), and on the nature of the optimization problem. In many applications, the value function can readily be shown to be continuous, even Lipschitz, but not C 1 in general. The finite horizon value function V (t, x) defined by (5) can be shown to be bounded and Lipschitz continuous under the following (rather strong) assumptions on the problem data: f, L, ψ are bounded with bounded first order derivatives. We shall assume this. It should be noted that in general it can happen that value functions fail to be con￾tinuous. In fact, the viscosity theory is capable of dealing with semicontinuous or even only locally bounded functions. Viscosity solution. Let us re-write the HJ equation (7) as follows: − ∂ ∂tV (t, x) + H(x, ∇xV (t, x)) = 0 in (t0, t1) × Rn , (7)0 with a new definition of the Hamiltonian H(x, λ) = sup v∈Rm {−λ · f(x, v) − L(x, v)} . (9)0 The sign convention used in (7)’ relates to the maximum principle in PDE, and is compat￾ible with the convention used for the general HJ equation (1). Note that the Hamiltonian is now convex in λ. A function V˜ ∈ C([t0, t1] × Rn ) is a viscosity subsolution (resp. supersolution) of (7)’ if for all φ ∈ C 1 ((t0, t1) × Rn ), − ∂ ∂tφ(s0, x0) + H(x0, ∇φ(s0, x0)) ≤ 0 (resp. ≥ 0) (54) at every point (s0, x0) where V˜ − φ attains a local maximum (resp. minimum). V˜ is a viscosity solution if it is both a subsolution and a supersolution. We now show that the value function V defined by (5) is a viscosity solution of (7)’. Subsolution property. Let φ ∈ C 1 ((t0, t1)×Rn ) and suppose that V −φ attains a local maximum at (s0, x0); so there exists r > 0 such that V (t, x) − φ(t, x) ≤ V (s0, x0) − φ(s0, x0) ∀ |x − x0| < r, |t − s0| < r. (55) Fix u(t) = u ∈ U for all t (constant control) and let ξ(·) denote the corresponding state trajectory with ξ(s0) = x0. By standard ODE estimates, we have |ξ(s0 + h) − x0| < r (56) for all 0 ≤ h ≤ h0 (some h0 > 0) - since U and f are bounded. Then by (55) V (s0 + h, ξ(s0 + h)) − φ(s0 + h, ξ(s0 + h)) ≤ V (s0, x0) − φ(s0, x0) (57) 15