Neurorobotics:From Vision to Action 62.3 The Role of the Cerebellum 1465 If a receptive field is excited,its response equals itself due to the fact that the control task itself was hidden the magnitude of a single adjustable weight specific within spinal cord and muscle models. to that receptive field.The CMAC output is the aver- age of the weights of the excited receptive fields.If The Schweighofer-Arbib Model nearby points in the input space excite the same re- The Schweighofer-Arbib model was introduced in ceptive fields,they produce the same output value.The Schweighofer [62.85].It does not use the CMAC state output only changes when the input crosses one of the encoder but tries to copy the anatomy of the cerebellum. receptive field boundaries.The Albus CMAC thus pro- All the cells,fibers,and axons in Fig.62.2a are included. duces piecewise-constant outputs.Learning takes place Several assumptions are made:(1)there are two types as described above. of mossy fibers,one type reflecting the desired state of CMAC neural networks have been applied in various the controlled plant and another which carries informa- control situations Miller [62.71],starting from adapta- tion on the current state.A mossy fiber diverges into tion of PID control parameters for an industrial robot approximately 16 branches;(2)granule cells have an arm and hand-eye systems up to biped walking (see average of four dendrites,each of which receive input Sabourin and Bruneau [62.82]). from different mossy fibers through a synaptic structure called the glomerulus;(3)three Golgi cells synapse on The Adjustable Pattern Generator APG a granule cell through the glomerulus and the strength of The APG model (Houk et al.[62.83])got its name be- their influence depends on the simulated geometric dis- cause the model can generate a burst command with tance between the glomerulus and the Golgi cell;(4)the adjustable intensity and duration.The APG is based on climbing fiber connection on nuclear cells as well as the same understanding of the mossy fiber-granule cell- deep nuclei is neglected. parallel fiber structure as CMAC,using the same state Learning in this model depends on directed error encoder,but has the crucial difference (Fig.62.2c)that information given by the climbing fibers from the infe- the role of the nuclei is crucial.In the APG model,each rior olive (IO).Here,long-term depression is performed nucleus cell is connected to a motor cell in a feedback when the IO firing rate provides an error signal for circuit.Activity in the loop is then modulated by Purk- an eligible synapse,while compensatory but slower in- inje cell inhibition,a modeling idea introduced by Arbib creases in synaptic strength can occur when no error etal.[62.84. signal is present.Schweighofer applied the model to ex- The learning algorithm determines which of the plain several acknowledged cerebellar system functions: PF-PC synapses will be updated in order to improve (1)saccadic eye movements,(2)two-link limb move- movement generation performance.This is the tradi- ment control (see Schweighofer et al.[62.86,87]),and tional credit assignment problem:which synapse (the (3)prism adaptation (Arbib et al.[62.881).Furthermore, structural credit assignment)must be updated based on control of a simulated human arm was demonstrated. a response issued when (temporal credit assignment). While the former is solved by the CFs,which are con- Multiple Paired Forward-Inverse Models sidered binary signals,for the latter eligibility traces on (MPFIM) the synapses are introduced,serving as memory for re- Building on a long history of cerebellar modeling, cent activity to determine which synapses are eligible Wolpert and Kawato [62.89]proposed a novel functional for updates.The motivation for the eligibility signal is model of the cerebellum which uses multiple coupled this:each firing of a PC cell will take some time to af- predictors and controllers which are trained for control, Part fect the animal's movement,and a further delay will each being responsible for a small state-space region. occur before the CF can signal an error in the move-The MPFIM model is based on the indirect/direct model ment in which the PC is involved.Thus the error signal approach by Kawato,and is also based on the microcom- should not affect those PF-PC synapses which are cur-plex theory.We noted earlier that a microzone is a group rently active,but should instead act upon those synapses of PCs,while a microcomplex combines the PCs of which affected the activity whose error is now being a microzone with their target cells in cerebellar nuclei. registered. In MPFIM,a microzone consists of a set of modules con- The APG has been applied in a few control situa- trolling the same degree of freedom and is learned by tions,e.g.,a single muscle-mass system and a simulated only one particular climbing fiber.The modules in this two-link robot arm.Unfortunately these applications do microzone compete to control this particular synergy. not allow us judge the performance of the APG scheme Inside such a module there are three types of PC whichNeurorobotics: From Vision to Action 62.3 The Role of the Cerebellum 1465 If a receptive field is excited, its response equals the magnitude of a single adjustable weight specific to that receptive field. The CMAC output is the aver￾age of the weights of the excited receptive fields. If nearby points in the input space excite the same re￾ceptive fields, they produce the same output value. The output only changes when the input crosses one of the receptive field boundaries. The Albus CMAC thus pro￾duces piecewise-constant outputs. Learning takes place as described above. CMAC neural networks have been applied in various control situations Miller [62.71], starting from adapta￾tion of PID control parameters for an industrial robot arm and hand–eye systems up to biped walking (see Sabourin and Bruneau [62.82]). The Adjustable Pattern Generator APG The APG model (Houk et al. [62.83]) got its name be￾cause the model can generate a burst command with adjustable intensity and duration. The APG is based on the same understanding of the mossy fiber–granule cell– parallel fiber structure as CMAC, using the same state encoder, but has the crucial difference (Fig. 62.2c) that the role of the nuclei is crucial. In the APG model, each nucleus cell is connected to a motor cell in a feedback circuit. Activity in the loop is then modulated by Purk￾inje cell inhibition, a modeling idea introduced by Arbib et al. [62.84]. The learning algorithm determines which of the PF–PC synapses will be updated in order to improve movement generation performance. This is the tradi￾tional credit assignment problem: which synapse (the structural credit assignment) must be updated based on a response issued when (temporal credit assignment). While the former is solved by the CFs, which are con￾sidered binary signals, for the latter eligibility traces on the synapses are introduced, serving as memory for re￾cent activity to determine which synapses are eligible for updates. The motivation for the eligibility signal is this: each firing of a PC cell will take some time to af￾fect the animal’s movement, and a further delay will occur before the CF can signal an error in the move￾ment in which the PC is involved. Thus the error signal should not affect those PF–PC synapses which are cur￾rently active, but should instead act upon those synapses which affected the activity whose error is now being registered. The APG has been applied in a few control situa￾tions, e.g., a single muscle–mass system and a simulated two-link robot arm. Unfortunately these applications do not allow us judge the performance of the APG scheme itself due to the fact that the control task itself was hidden within spinal cord and muscle models. The Schweighofer–Arbib Model The Schweighofer–Arbib model was introduced in Schweighofer [62.85]. It does not use the CMAC state encoder but tries to copy the anatomy of the cerebellum. All the cells, fibers, and axons in Fig. 62.2a are included. Several assumptions are made: (1) there are two types of mossy fibers, one type reflecting the desired state of the controlled plant and another which carries informa￾tion on the current state. A mossy fiber diverges into approximately 16 branches; (2) granule cells have an average of four dendrites, each of which receive input from different mossy fibers through a synaptic structure called the glomerulus; (3) three Golgi cells synapse on a granule cell through the glomerulus and the strength of their influence depends on the simulated geometric dis￾tance between the glomerulus and the Golgi cell; (4) the climbing fiber connection on nuclear cells as well as deep nuclei is neglected. Learning in this model depends on directed error information given by the climbing fibers from the infe￾rior olive (IO). Here, long-term depression is performed when the IO firing rate provides an error signal for an eligible synapse, while compensatory but slower in￾creases in synaptic strength can occur when no error signal is present. Schweighofer applied the model to ex￾plain several acknowledged cerebellar system functions: (1) saccadic eye movements, (2) two-link limb move￾ment control (see Schweighofer et al. [62.86, 87]), and (3) prism adaptation (Arbib et al. [62.88]). Furthermore, control of a simulated human arm was demonstrated. Multiple Paired Forward-Inverse Models (MPFIM) Building on a long history of cerebellar modeling, Wolpert and Kawato [62.89] proposed a novel functional model of the cerebellum which uses multiple coupled predictors and controllers which are trained for control, each being responsible for a small state-space region. The MPFIM model is based on the indirect/direct model approach by Kawato, and is also based on the microcom￾plex theory. We noted earlier that a microzone is a group of PCs, while a microcomplex combines the PCs of a microzone with their target cells in cerebellar nuclei. In MPFIM, a microzone consists of a set of modules con￾trolling the same degree of freedom and is learned by only one particular climbing fiber. The modules in this microzone compete to control this particular synergy. Inside such a module there are three types of PC which Part G 62.3