KIM AND LEWIS: OPTIMAL DESIGN OF NEURAL-NETWORK CONTROLLER and V(a, t) is referred to as the value function. It satisfies the where h(er) is given by(12). It is referred to as an optima partial differential equation computed torque controller(OCTC) av(e,t=L(i, u, )+222.(25)B Stability Analysis Theorem 2: Suppose that matrices K and A exist that satisfy The minimum is attained for the optimal control u(t)=w(t), the hypotheses of Lemma 1, and in addition, there exist con- and the Hamiltonian is then given by stants K,i and k2 such that 0 <hi< k2 < oo, and the spectrum H*=minL(, u)+ov(, t) of P is bounded in the sense that hiI <P< h2I on(to, oo) Then using the feedback control in(29)and(20)results in the controlled nonlinear system H(乏,u* av(a, t) it=A(,,i-B(R B(g PIt). (35) V(2,t) This is globally exponentially stable(GES)regarding the origi 92n Lemma 1: The following function V composed of z, M(g) Proof: The quadratic function v(i, t )is a suitable and a positive symmetric matrix K=K E9pnxn satisfies the Lyapunov function candidate, because it is positive radially H-J-B equation growing with z. It is continuous and has a unique minimum at V=是Pq)=kO2×n 2> the origin of the error space. It remains to show that dv/dt<O (27) for all 2#0. From the solution of the H-J-B equation (A12), it follows that where K and A in(10)and(27) can be found from the riccati differential equation L(2,). PA+APf-PBRBP+P+Q=Onxn.(28)Substituting(29)for(31)gives The optimal control u(t)that minimizes(21) subject to(20)is d(,t)~{2Q+(BP2)r-1(BP2}<0 ut()=-1BP(q)=-Rrt).(29) Vt>0≠0 (37) See Appendix A for proof. The time derivative of the lyapunov function is negative defl Theorem 1: Let the symmetric weighting matrices Q, R be nite, and the assertion of the theorem then follows directly from chosen such that the properties of the Lyapunov function [9] Q Qu Q 75 R=Q2(30) IV CMAC NEURAL-CONTROLLER DESIGN with Q12+Q12<nxn. Then the K and A required in Le The block diagram in Fig. 2 shows the major components I can be determined from the following relations that embody the CMAC neural controller. The external-control torques to the joints are composed of the optimal-feedback con- 1(Q12+QT2)>0nXn (31) trol law given in Theorem I plus the CMAC neural-network The nonlinear robot function can be represented by a CMAC eural network h(x)=W2y(x)+E(x)l|=(x)川≤EM(38) with(32)solved for A using Lyapunov equation solvers(e.g MatLab[15D) where p(sr)is a multidimensional receptive-field function for See Appendix B for proof. the CMAc Remarks Then a functional estimate h(r)of h(r)can be written as 1)In order to guarantee positive definiteness of the con- structed matrix Q, the following inequality [7] must be h(a)=w plr) Aim(Q2)>|Ql2/am(Q1).(3) r(t)=Wry(x)-'(t)-v() 2)With the optimal-feedback control law c(t)calculated using Theorem 1, the torques r(t) to apply to the robotic where u(t) is a robustifying vector. Then(11)becomes system are calculated according to the control input Mr(t)=-Vmr(t)+Wp(=)+E(a)+Ta(t) T*(t)=(x)-"(t +u(t)+u(tKIM AND LEWIS: OPTIMAL DESIGN OF NEURAL-NETWORK CONTROLLER 25 and is referred to as the value function. It satisfies the partial differential equation (25) The minimum is attained for the optimal control , and the Hamiltonian is then given by (26) Lemma 1: The following function composed of , and a positive symmetric matrix satisfies the H–J–B equation: (27) where and in (10) and (27) can be found from the Riccati differential equation (28) The optimal control that minimizes (21) subject to (20) is (29) See Appendix A for proof. Theorem 1: Let the symmetric weighting matrices , be chosen such that (30) with . Then the and required in Lemma 1 can be determined from the following relations: (31) (32) with (32) solved for using Lyapunov equation solvers (e.g., MatLab [15]). See Appendix B for proof. Remarks: 1) In order to guarantee positive definiteness of the con￾structed matrix , the following inequality [7] must be satisfied (33) 2) With the optimal-feedback control law calculated using Theorem 1, the torques to apply to the robotic system are calculated according to the control input (34) where is given by (12). It is referred to as an optimal￾computed torque controller (OCTC). B. Stability Analysis Theorem 2: Suppose that matrices and exist that satisfy the hypotheses of Lemma 1, and in addition, there exist con￾stants and such that , and the spectrum of is bounded in the sense that on . Then using the feedback control in (29) and (20) results in the controlled nonlinear system (35) This is globally exponentially stable (GES) regarding the origin in . Proof: The quadratic function is a suitable Lyapunov function candidate, because it is positive radially, growing with . It is continuous and has a unique minimum at the origin of the error space. It remains to show that for all . From the solution of the H–J–B equation (A12), it follows that (36) Substituting (29) for (31) gives (37) The time derivative of the Lyapunov function is negative defi￾nite, and the assertion of the theorem then follows directly from the properties of the Lyapunov function [9]. IV. CMAC NEURAL-CONTROLLER DESIGN The block diagram in Fig. 2 shows the major components that embody the CMAC neural controller. The external-control torques to the joints are composed of the optimal-feedback con￾trol law given in Theorem 1 plus the CMAC neural-network output components. The nonlinear robot function can be represented by a CMAC neural network (38) where is a multidimensional receptive-field function for the CMAC. Then a functional estimate of can be written as (39) The external torque is given by (40) where is a robustifying vector. Then (11) becomes (41)