EEE TRANSACTIONS ON SYSTEMS. MAN. AND CYBERNETICS-PART C: APPLICATIONS AND REVIEWS VOL 30. NO. 1. FEBUARY 2000 Then, an estimate of f(a)can be given by Property 1--Inertia: The inertia matrix M(g) is uniformly bounded f(r) m1I≤M(q)≤m2Im,m2>0andI∈我n,(l6) where W are estimates of the ideal weight values. The Lya punov method is applied to derive reinforcement adaptive Property 2-Skeww Symmetry: The matrix learning rules for the weight values. Since these adaptive learning rules are formulated from the stability analysis of the N(q立=M(q)-2Vm(g立 controlled system, the system performance can be guaranteed or closed-loop control s skew-symmetric Robot Arm Dynam II. OPTIMAL-COMPUTED TORQUE-CONTROLLER DESIGN The dynamics of an n-link robot manipulator may be ex- A.H-J-B Optimization pressed in the Lagrange form [91 Define the velocity-error dynamics M(q)+Vm(q立+F+fc(①)+gq)+7(t)=T(t et)=-Ae(t q(t)∈我 variable The following augmented system is obtained M(q)∈ inertia Vm(q),q)∈我× Coriolis/centripetal forces = gq∈究 gravitational force L+」=[0nxm-M-v M-1 u(ty diagonal matrix of viscous friction co- or with shorter notation efficients. ∫c(i)∈我 Coulomb friction coefficient x(1)=A(q,q)(t)+B(q)u(t) external disturbances The external control torque to each joint is t(tEgen with A(g立∈我x如,B(q)∈我2n,and(t)∈界nx Given a desired trajectory ga(t)E R", the tracking errors are i(t)is defined as it)T=[etT r(tT].A quadratic perfor e(t)=qa(t-t) and e(t)=qa(t)-i(t) (9 mance index J(u) is as follows and the instantaneous performance measure is defined L(E, u)dt rt=et)+ Aet) (10) with the Lagrangian where AE R xn is the constant-gain matrix or critic (not nec- L(E, u)=22(+Qa(t)+3uT(t)Ru(t) The robot dynamics( 8)may be written as 1Q121「e Qi2 Q,r+2 uTRu(22) M(q(t)=-Vmn(,@)r(t)-T(t)+(x)(11) Given the performance index J(u), the control objective is where the robot nonlinear function is to find the auxiliary control input u(t)that minimizes(21)sub. ject to the differential constraints imposed by (19). The optimal h(r)=M(a(d+ Ae+Vm(a, i a+Ae) control that achieves this objective will be denoted by u (t). It is Fq+f(q)+9(q)+7(t) (12) worth noting for now, that only the part of the control-input-to- robotic-system denoted by u(t) in(14) is penalized. This is rea and. for instance sonable from a practical standpoint, since the gravity, Coriolis, and friction-compensation terms in(12)cannot be modified by (t)=[ereq效] (13)the optimal-design phase A necessary and sufficient condition for u(t) to minimize This key function h(r)captures all the unknown dynamics of (21)subject to(20)is that there exist a function V=v(a, ty the robot arm satisfying the H-J-B equation Now define a control-input torque as o(t+milr(2u一 (2,t) (t)=b(x)-u(t) with u(t)E an auxiliary control input to be optimized later. where the Hamiltionian of optimization is defined as The closed-loop system becomes Mart=-vm(, or(t)+ut (15) H av(E, t) L(2,v)+(t24 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 30, NO. 1, FEBUARY 2000 Then, an estimate of can be given by (7) where are estimates of the ideal weight values. The Lya￾punov method is applied to derive reinforcement adaptive learning rules for the weight values. Since these adaptive learning rules are formulated from the stability analysis of the controlled system, the system performance can be guaranteed for closed-loop control. B. Robot Arm Dynamics and Properties The dynamics of an -link robot manipulator may be ex￾pressed in the Lagrange form [9] (8) with joint variable; inertia; Coriolis/centripetal forces; gravitational forces; diagonal matrix of viscous friction co￾efficients; Coulomb friction coefficients; external disturbances. The external control torque to each joint is . Given a desired trajectory , the tracking errors are and (9) and the instantaneous performance measure is defined as (10) where is the constant-gain matrix or critic (not nec￾essarily symmetric). The robot dynamics (8) may be written as (11) where the robot nonlinear function is (12) and, for instance (13) This key function captures all the unknown dynamics of the robot arm. Now define a control-input torque as (14) with an auxiliary control input to be optimized later. The closed-loop system becomes (15) Property 1—Inertia: The inertia matrix is uniformly bounded and (16) Property 2—Skew Symmetry: The matrix (17) is skew-symmetric. III. OPTIMAL-COMPUTED TORQUE-CONTROLLER DESIGN A. H–J–B Optimization Define the velocity-error dynamics (18) The following augmented system is obtained: (19) or with shorter notation (20) with , , and . is defined as . A quadratic perfor￾mance index is as follows: (21) with the Lagrangian (22) Given the performance index , the control objective is to find the auxiliary control input that minimizes (21) sub￾ject to the differential constraints imposed by (19). The optimal control that achieves this objective will be denoted by . It is worth noting for now, that only the part of the control-input-to￾robotic-system denoted by in (14) is penalized. This is rea￾sonable from a practical standpoint, since the gravity, Coriolis, and friction-compensation terms in (12) cannot be modified by the optimal-design phase. A necessary and sufficient condition for to minimize (21) subject to (20) is that there exist a function satisfying the H–J–B equation [10] (23) where the Hamiltionian of optimization is defined as (24)