KIM AND LEWIS: OPTIMAL DESIGN OF NEURAL-NETWORK CONTROLLER Fig. 6. Performance of CMAC neural-network controller (40):(a) tracking error with disturbance and friction for fast motion and(b)tracking error of mass variation(m2; 2.3-4.0 kg at 5 s, m2;4.0- 2.3 kg at 12 s)with disturbance and friction for fast motion(solid: joint 1, dotted: joint 2) ence of modeling uncertainties and frictional forces. The salient matrix whose elements are partial derivatives of the elements of feature of the CMAC neural-HjB design is that the control ob- Pw.r.t. ei jective is obtained with completely unknown nonlinearities in A candidate for the Hamiltonian H(24)is the sum of(A5) the robotic manipulator. The proposed neural-adaptive learning and the Lagrangian(22). Now we are ready to evaluate how H shows both robustness and adaptation to changing system dy- depends on ut)E en. The ut)=u(t) for which H has namics. To that end, a critic signal is incorporated into the adap- its minimum values is obtained from the partial derivative w.r.t tive-learning scheme. The application potential of the proposed u(t) methodology lies in the control design in areas such as robotics Since ut) is unconstrained, ( A3)requires that and flight control and in motion-control analysis(e.g, of biome chanics) aH V(2,t) 0 APPENDIX A PROOF OF LEMMA 1 which gives a candidate for the optimal control The theorem claims that the HJB equation V(,t) aNa2n=min1(2)+V(么,t (A8) (A1) sInce is satisfied for a function BcH =22P@2K Oxn z We know that(A3)is satisfied by u(t), given(A8). Inserting where .6)into(A8)giv mL(,v)+( =H( (A3) u(t=-RB P(g)i (A10) To derive optimal-control law, the partial derivatives of the Notice that the relation function v need to be evaluated Here, we have the time deriva- tive of the function v DB=[D1 XOnxn +O2mxn XM-I(q]=O2nx2n(All) is used The gradient of v with respect to the error state i is A necessary and sufficient condition for optimality is that the chosen value function V satisfies(23). Substituting(24)for(23) ID eld av(, t) av(a, t) 22+1(2,t) aP P where it is understood that the partial derivatives of V in(Al (A6)are being evaluated along the optimal control u(t).Inserting (A4)into(A12), in(A6, Dhas dimension2n×2n,O2n×lisa2n×1 zero vector, and the not aP(g/aei is used to represent the 2n X 2n zP(q)2+2Pq+L(,x)=0.(A13)KIM AND LEWIS: OPTIMAL DESIGN OF NEURAL-NETWORK CONTROLLER 29 (a) (b) Fig. 6. Performance of CMAC neural-network controller (40): (a) tracking error with disturbance and friction for fast motion and (b) tracking error of mass variation (m ; 2:3 ! 4:0 kg at 5 s, m ; 4:0 ! 2:3 kg at 12 s) with disturbance and friction for fast motion (solid: joint 1, dotted: joint 2). ence of modeling uncertainties and frictional forces. The salient feature of the CMAC neural-HJB design is that the control ob￾jective is obtained with completely unknown nonlinearities in the robotic manipulator. The proposed neural-adaptive learning shows both robustness and adaptation to changing system dy￾namics. To that end, a critic signal is incorporated into the adap￾tive-learning scheme. The application potential of the proposed methodology lies in the control design in areas such as robotics and flight control and in motion-control analysis (e.g., of biome￾chanics). APPENDIX A PROOF OF LEMMA 1 The theorem claims that the HJB equation (A1) is satisfied for a function (A2) where (A3) To derive optimal-control law, the partial derivatives of the function need to be evaluated. Here, we have the time deriva￾tive of the function (A4) The gradient of with respect to the error state is (A5) with (A6) In (A6), has dimension is a zero vector, and the notation is used to represent the matrix whose elements are partial derivatives of the elements of w.r.t. . A candidate for the Hamiltonian (24) is the sum of (A5) and the Lagrangian (22). Now we are ready to evaluate how depends on . The for which has its minimum values is obtained from the partial derivative w.r.t. . Since is unconstrained, (A3) requires that (A7) which gives a candidate for the optimal control (A8) since (A9) We know that (A3) is satisfied by , given (A8). Inserting (A5) and (A6) into (A8) gives (A10) Notice that the relation (A11) is used. A necessary and sufficient condition for optimality is that the chosen value function satisfies (23). Substituting (24) for (23) yields (A12) where it is understood that the partial derivatives of in (A12) are being evaluated along the optimal control . Inserting (A4) into (A12), we obtain (A13)