The problem is illustrated in Figure 1. In Fig. la, a possible trajectory of the state variable with the initial value x is depicted. If the trajectory of the control vector is specified for the entire time horizon [o, t,I, the trajectory of the state variable is completely characterized. The value of the state variable at time t and the choice of the control vector then jointly determine fo(x(o),u(o),t) In Fig. lb we graph the part of the value of the objective functional which has been realized at any time t for the particular trajectory of the control vector fo therefore, appears as the slope in Fig 1b, while the value of the objective functional the sum of the integral from fo to t,, of fo, and So, the scrap value at terminal time Our problem is to obtain the trajectory of the control vector that maximizes the The major difficulty of this problem lies in the fact that an entire time path of the control vector must be chosen. This amounts to a continuously infinite number of control variables. In other words, what must be found is not just the optimal numbers but the optimal functions. The basic idea of control theory is to transform the problem hal uing the entire optimal path of control variables into the problem of find ing the optimal values of control variables at each instant of time. In this way the problem of choosing an infinite number of variables is decomposed into an infinite number of mor elementary problems each of which involves determining a finite number of variables The objective functional can be broken into three pieces for any time t-a past, a t and a futu f0(x(),u(n),)dn ∫f6(x0):()d f0(x(0),u(),)dr+S0(x(1),1) The decisions taken at any time have two effects. They directly affect the present erm fo(x(t), u(t), !)dt by changing fo. They also change x, and hence the future path of x(o), through i=f(x(o), u(t),t). The new path of x(o) changes the future part of the functional For example, if a firm increases investment at time t, the rate at which profits are earned at that time falls because the firm must pay for the investment. The investment however, increases the amount of capital available in the future and therefore profits earned in the future. The firm must make investment decisions weighing these two effects. In general, the choice of the control variables at any instant of time must take into account both the instantaneous effect on the current earnings foAt and the indirect effect on the future earnings [ fodr'+So through a change in the state 192Appendix IV 192 The problem is illustrated in Figure 1. In Fig.la, a possible trajectory of the state variable with the initial value 0 x is depicted. If the trajectory of the control vector is specified for the entire time horizon [ ] 0 1 t ,t , the trajectory of the state variable is completely characterized. The value of the state variable at time t and the choice of the control vector then jointly determine ( ( ), ( ), ) 0 f x t u t t . In Fig.1b we graph the part of the value of the objective functional which has been realized at any time t for the particular trajectory of the control vector 0 f , therefore, appears as the slope in Fig.1b, while the value of the objective functional is the sum of the integral from 0 t to 1 t , of 0 f , and S0 , the scrap value at terminal time. Our problem is to obtain the trajectory of the control vector that maximizes the objective functional. The major difficulty of this problem lies in the fact that an entire time path of the control vector must be chosen. This amounts to a continuously infinite number of control variables. In other words, what must be found is not just the optimal numbers but the optimal functions. The basic idea of control theory is to transform the problem of choosing the entire optimal path of control variables into the problem of finding the optimal values of control variables at each instant of time. In this way the problem of choosing an infinite number of variables is decomposed into an infinite number of more elementary problems each of which involves determining a finite number of variables. The objective functional can be broken into three pieces for any time t - a past, a present and a future - : ò ò ò +D +D + ¢ ¢ ¢ ¢+ + ¢ ¢ ¢ ¢ = ¢ ¢ ¢ ¢ 1 0 ( ( ) , ( ) , ) ( ( ), ). ( ( ) , ( ) , ) ( ( ) , ( ) , ) 0 0 1 1 0 0 t t t t t t t t f x t u t t dt S x t t f x t u t t dt J f x t u t t dt The decisions taken at any time have two effects. They directly affect the present term, ò +D ¢ ¢ ¢ ¢ t t t f (x(t) ,u(t) ,t )dt 0 , by changing 0 f . They also change x& , and hence the future path of x(t) , through ( ( ), ( ), ) 1 x& = f x t u t t . The new path of x(t) changes the future part of the functional. For example, if a firm increases investment at time t, the rate at which profits are earned at that time falls because the firm must pay for the investment. The investment, however, increases the amount of capital available in the future and therefore profits earned in the future. The firm must make investment decisions weighing these two effects. In general, the choice of the control variables at any instant of time must take into account both the instantaneous effect on the current earnings f Dt 0 and the indirect effect on the future earnings ò +D ¢+ 1 0 0 t t t f dt S through a change in the state