第5章 Perspectives on Fuzzy Control 教学内容 本章讲述模糊控制与传统控制的相互关系、模糊神经网络控制、模糊控制和遗传算法的相互联系与区别、 模糊控制与基于知识系统的交叉和融合。 教学重点 重点内容是讲述模糊控制与传统控制的相互关系,模糊系统与神经网络、遗传算法、基于知识的学习系 统的相互联系以及交叉融合。 教学难点 对模糊控制与传统控制、神经网络控制、遗传算法以及基于知识的系统的区别和联系的准确把握和理 解,学会用发展和联系的观点看待智能控制学科的发展。 教学要求 要求学生基本了解模糊控制与传统控制、神经网络控制、遗传算法以及基于知识的系统的区别和联系. 通过本章的学习,进一步加深对智能控制特点的理解,学会用发展和联系的观点看待智能控制学科的发展 从而培养学生创常新意识和能力,形成正确的工作方法。 5.1 Overview Fuzzy control does not exist as an isolated topic devoid of relationships to other fields, and it is important to understand how it relates to these other fields in order to strengthen your understanding of it. We have emphasized that fuzzy control has its foundations in conventional control and that there are many relationships to techniques, ideas, and methodologies there. Fuzzy control is also an"intelligent control"technique, and hence there are certain relationships between it and other intelligent control methods. In this chapter we will provide a brief overview of some of the basic relationships between fuzzy control and other control methods. This will give the reader who has a good understanding of fuzzy control a glimpse of related topics in other areas. Moreover, it will give the reader who has a good understanding of other areas of control an idea of what the field of fuzzy control is concerned with We begin the chapter in Section 5.2 by providing a conventional control engineering perspective on fuzzy control This is essentially a summary of many of the points that we have made throughout the text, but here we bring them all together. Following this, in Section 5.3 we introduce two popular areas in neural networks, the multilayer perceptron and the radial basis function neural network. We explain that a class of radial basis function neural networks is identical to a class of fuzzy systems. Moreover, we explain how techniques covered in this book(e.g, gradient training and adaptive control) can be used for neural networks. In Section 5. 4 we explain genetic algorithms, their relationship to the field of control, and particularly their use with fuzzy systems. Next, in Section 5.5 we provide an overview of some of the relationships to knowledge-based systems, particularly expert systems(and hence expert control) and planning systems
第 5 章 Perspectives on Fuzzy Control 教学内容 本章讲述模糊控制与传统控制的相互关系、模糊神经网络控制、模糊控制和遗传算法的相互联系与区别、 模糊控制与基于知识系统的交叉和融合。 教学重点 重点内容是讲述模糊控制与传统控制的相互关系，模糊系统与神经网络、遗传算法、基于知识的学习系 统的相互联系以及交叉融合。 教学难点 对模糊控制与传统控制、神经网络控制、遗传算法以及基于知识的系统的区别和联系的准确把握和理 解，学会用发展和联系的观点看待智能控制学科的发展。 教学要求 要求学生基本了解模糊控制与传统控制、神经网络控制、遗传算法以及基于知识的系统的区别和联系。 通过本章的学习，进一步加深对智能控制特点的理解，学会用发展和联系的观点看待智能控制学科的发展， 从而培养学生创常新意识和能力，形成正确的工作方法。 5.1 Overview Fuzzy control does not exist as an isolated topic devoid of relationships to other fields, and it is important to understand how it relates to these other fields in order to strengthen your understanding of it. We have emphasized that fuzzy control has its foundations in conventional control and that there are many relationships to techniques, ideas, and methodologies there. Fuzzy control is also an "intelligent control" technique, and hence there are certain relationships between it and other intelligent control methods. In this chapter we will provide a brief overview of some of the basic relationships between fuzzy control and other control methods. This will give the reader who has a good understanding of fuzzy control a glimpse of related topics in other areas. Moreover, it will give the reader who has a good understanding of other areas of control an idea of what the field of fuzzy control is concerned with. We begin the chapter in Section 5.2 by providing a conventional control engineering perspective on fuzzy control. This is essentially a summary of many of the points that we have made throughout the text, but here we bring them all together. Following this, in Section 5.3 we introduce two popular areas in neural networks, the multilayer perceptron and the radial basis function neural network. We explain that a class of radial basis function neural networks is identical to a class of fuzzy systems. Moreover, we explain how techniques covered in this book (e.g., gradient training and adaptive control) can be used for neural networks. In Section 5.4 we explain genetic algorithms, their relationship to the field of control, and particularly their use with fuzzy systems. Next, in Section 5.5 we provide an overview of some of the relationships to knowledge-based systems, particularly expert systems (and hence expert control) and planning systems
Finally, in Section 5.6 we provide an overview of the general area of (hierarchical) intelligent and autonomous control where we offer some ideas on how to define the field of intelligent control and how some of the most general intelligent controllers operate. We use an "intelligent vehicle highway system"problem to illustrate the use of the intelligent autonomous controller functional architecture This chapter is meant to provide a view of, and motivation for, the main areas in the field of intelligent control. The reader interested only in fuzzy control can certainly ignore this chapter; we do not, however, advise this as the relationships to other fields often suggest ideas on how to expand the basic fuzzy control methods and may provide key deas on how to solve a control problem for a particular application 5.2 Fuzzy Versus Conventional Control What are the advantages and disadvantages of fuzzy control as compared to conventional control? What are the perspectives of conventional control engineers on fuzzy control? In this section we will attempt to give answers to these questions by asking, and at least partially answering, a series of questions that we have accumulated over the years from a variety of engineers in industry and universities concerned about whether to use fuzzy or conventional control. We break the questions into three categories and use the questions to summarize several points made in earlier chapters 5.2.1 Modeling Issues and Design Methodology First, we will discuss several issues related to modeling and the overall fuzzy controller design methodology Is the fuzzy controller design methodology viable? Success in a variety of applications (.g, the flexible-link rob application studied in this book)has proven fuzzy control to be a viable methodology and therefore worthy of consideration 2. Do engineers like the methodology? Some do, and some do not. Engineers who have found success with it tend to like it. Often, we find that if engineers invest the time into learning it, they find it to be a tool with which they are comfortable working(they feel like it is"one more tool in their toolbox"). This may be because fuzzy systems are interpolators and engineers are used to thinking about using interpolation as a solution to a wide variety of problems 3. Will the methodology always work? No. The reason we can be so definite in this answer is that it is not the methodology that ultimately leads to success; it is the clever ideas that the control engineer uses to achieve high-performance control. Fuzzy control is a vehicle, and the engineer is the driver. Some find that the vehicle is comfortable and that they can coax it into performing all kinds of functions for them. Others are not so comfortable with 4. Does the design methodology always shorten the"lead time"to design and implementation? In talking with many people in industry, we have found that most often it does(and this is very important, especially in today's competitive climate), but we have also heard of instances where people factor in the cost of having their engineers learn the method and then found the membership functions very hard to tune. In these cases the clear answer from the engineers was that it did not make things easier. We have heard from some that fuzzy logic implements, in a similar way, the standard logic and interpolation methods they already use. Sometimes such engineers find that the fuzzy control jargon clouds the issues that are central to the control problem. Others like that it helps to formalize what they have been doing and helps to suggest ideas for other approaches 5. Is a model used in the fuzzy control design methodology? It is possible that a mathematical model is not used However, often it is used in simulation to redesign a fuzzy controller. Others argue that a model is always used: even if it
Finally, in Section 5.6 we provide an overview of the general area of (hierarchical) intelligent and autonomous control where we offer some ideas on how to define the field of intelligent control and how some of the most general intelligent controllers operate. We use an "intelligent vehicle highway system" problem to illustrate the use of the intelligent autonomous controller functional architecture. This chapter is meant to provide a view of, and motivation for, the main areas in the field of intelligent control. The reader interested only in fuzzy control can certainly ignore this chapter; we do not, however, advise this as the relationships to other fields often suggest ideas on how to expand the basic fuzzy control methods and may provide key ideas on how to solve a control problem for a particular application. 5.2 Fuzzy Versus Conventional Control What are the advantages and disadvantages of fuzzy control as compared to conventional control? What are the perspectives of conventional control engineers on fuzzy control? In this section we will attempt to give answers to these questions by asking, and at least partially answering, a series of questions that we have accumulated over the years from a variety of engineers in industry and universities concerned about whether to use fuzzy or conventional control. We break the questions into three categories and use the questions to summarize several points made in earlier chapters. 5.2.1 Modeling Issues and Design Methodology First, we will discuss several issues related to modeling and the overall fuzzy controller design methodology. 1. Is the fuzzy controller design methodology viable? Success in a variety of applications (e.g., the flexible-link robot application studied in this book) has proven fuzzy control to be a viable methodology and therefore worthy of consideration. 2. Do engineers like the methodology? Some do, and some do not. Engineers who have found success with it tend to like it. Often, we find that if engineers invest the time into learning it, they find it to be a tool with which they are comfortable working (they feel like it is "one more tool in their toolbox"). This may be because fuzzy systems are interpolators and engineers are used to thinking about using interpolation as a solution to a wide variety of problems. 3. Will the methodology always work? No. The reason we can be so definite in this answer is that it is not the methodology that ultimately leads to success; it is the clever ideas that the control engineer uses to achieve high-performance control. Fuzzy control is a vehicle, and the engineer is the driver. Some find that the vehicle is comfortable and that they can coax it into performing all kinds of functions for them. Others are not so comfortable with it. 4. Does the design methodology always shorten the "lead time" to design and implementation? In talking with many people in industry, we have found that most often it does (and this is very important, especially in today's competitive climate), but we have also heard of instances where people factor in the cost of having their engineers learn the method and then found the membership functions very hard to tune. In these cases the clear answer from the engineers was that it did not make things easier. We have heard from some that fuzzy logic implements, in a similar way, the standard logic and interpolation methods they already use. Sometimes such engineers find that the fuzzy control jargon clouds the issues that are central to the control problem. Others like that it helps to formalize what they have been doing and helps to suggest ideas for other approaches. 5. Is a model used in the fuzzy control design methodology? It is possible that a mathematical model is not used. However, often it is used in simulation to redesign a fuzzy controller. Others argue that a model is always used: even if it
is not written down, some type of model is used"in your head. 6. Since most people claim that no formal model is used in the fuzzy control design methodology, the following questions a(a) Is it not true that there are few, if any, assumptions to be violated by fuzzy control and that the technique can be indiscriminately applied? Yes, and sometimes it is applied to systems where it is clear that a Pid controller or look-up table would be just as effective. So, if this is the case, then why not use fuzzy control? Because it is more computationally complex than a Pid controller and the pid controller is much more widely (b) Are heuristics all that are available to perform fuzzy controller design? No. any good models that can be used, probably should be (c) By ignoring a formal model, if it is available, is it not the case that a significant amount of information about how to control the plant is ignored? Yes. If, for example, you have a model of a complex process, we ofter use simulations to gain an understanding of how best to control the plant-and this knowledge can be used to design a fuzzy controller a(d) Can standard control theoretic analysis be used to verify the operation of the resulting control system? Sometimes, if the fuzzy control system satisfies the assumptions needed for the mathematical analysis. This will (e) Will it be difficult to clearly characterize the limitations of various fuzzy control techniques(i.e, to classify which plants can be controlled best with different fuzzy or conventional controllers)? Yes (f) Will it be difficult to clearly relate the results of using the fuzzy controller to previous work in conventional control to definitively show that contributions are being made to the field of control? Yes 7. Is there always a formal model available for control design? No, but for most systems there is at least an approximate model available. This information is often valuable and should not be ignore 8. Does the use of fuzzy controllers limit the design methodology as compared to the use of more general expert controllers? Expert controllers use more general knowledge-representation schemes and inference strategies(see more details in Section 5.5.1), so for some plants it may be advantageous to use the expert controller. It is, however, not clear at this point what class of plants call for the use of expert control 5.2.2 Stability and Performance analysis Next, we will discuss several issues related to the performance analysis of fuzzy control systems 1 Is verification and certification of fuzzy control systems important? Yes, especially for safety-critical systems(e. g an aircraft). It may not be as important for certain applications(e. g, a washing machine with a fuzzy control system) 2. What are the roles of simulation and implementation in evaluating the performance of fuzzy control systems? They lay exactly the same role as for conventional control systems 3. What are the roles of the following nonlinear analysis approaches in fuzzy control system design? (a) Phase plane analy (b) Describing function analysis (c) Stability analysis: Lyapunov's first and second methods; absolute stability; and the small gain theorem (d) Analysis of steady-state errors (e) Method of equivalent gains (f) Cell-to-cell mapping approaches
is not written down, some type of model is used "in your head." 6. Since most people claim that no formal model is used in the fuzzy control design methodology, the following questions arise: (a) Is it not true that there are few, if any, assumptions to be violated by fuzzy control and that the technique can be indiscriminately applied? Yes, and sometimes it is applied to systems where it is clear that a PID controller or look-up table would be just as effective. So, if this is the case, then why not use fuzzy control? Because it is more computationally complex than a PID controller and the PID controller is much more widely understood. (b) Are heuristics all that are available to perform fuzzy controller design? No. Any good models that can be used, probably should be. (c) By ignoring a formal model, if it is available, is it not the case that a significant amount of information about how to control the plant is ignored? Yes. If, for example, you have a model of a complex process, we often use simulations to gain an understanding of how best to control the plant—and this knowledge can be used to design a fuzzy controller. (d) Can standard control theoretic analysis be used to verify the operation of the resulting control system? Sometimes, if the fuzzy control system satisfies the assumptions needed for the mathematical analysis. This will be discussed in more detail in the next section. (e) Will it be difficult to clearly characterize the limitations of various fuzzy control techniques (i.e., to classify which plants can be controlled best with different fuzzy or conventional controllers)? Yes. (f) Will it be difficult to clearly relate the results of using the fuzzy controller to previous work in conventional control to definitively show that contributions are being made to the field of control? Yes. 7. Is there always a formal model available for control design? No, but for most systems there is at least an approximate model available. This information is often valuable and should not be ignored. 8. Does the use of fuzzy controllers limit the design methodology as compared to the use of more general expert controllers? Expert controllers use more general knowledge-representation schemes and inference strategies (see more details in Section 5.5.1), so for some plants it may be advantageous to use the expert controller. It is, however, not clear at this point what class of plants call for the use of expert control. 5.2.2 Stability and Performance Analysis Next, we will discuss several issues related to the performance analysis of fuzzy control systems. 1. Is verification and certification of fuzzy control systems important? Yes, especially for safety-critical systems (e.g., an aircraft). It may not be as important for certain applications (e.g., a washing machine with a fuzzy control system). 2. What are the roles of simulation and implementation in evaluating the performance of fuzzy control systems? They play exactly the same role as for conventional control systems. 3. What are the roles of the following nonlinear analysis approaches in fuzzy control system design? (a) Phase plane analysis. (b) Describing function analysis. (c) Stability analysis: Lyapunov's first and second methods; absolute stability; and the small gain theorem. (d) Analysis of steady-state errors. (e) Method of equivalent gains. (f) Cell-to-cell mapping approaches
Several of these approaches may apply to the analysis of the behavior of the fuzzy control system you design 4. What are the problems with utilizing mathematical analysis for fuzzy control system verification? The techniques take time to learn. The problems for which fuzzy control are particularly well suited, and where there is often very good motivation to use fuzzy rather than conventional control, are the control problems where the plant has complex nonlinear behavior,and where a model is hard to derive due to inherent uncertainties. Each of these characteristics often makes the assumptions that are needed for the nonlinear analysis techniques invalid, so the theory often does not end up offering much when it is really needed 5. Does fuzzy control provide"robust control"? If so, can this be demonstrated mathematically or experimentally? There has been a recent focus in research on stability analysis to show that fuzzy control does provide robust control. It is very difficult, of course, to show robustness via experimentation since by its very definition robustness verification requires extensive experimentation(e.g, you could not call the fuzzy controller for the rotational inverted pendulum case or"robust"when it was only shown to be successful for one disturbance condition) 5.2.3 Implementation and general Issues Finally, we will discuss several issues related to implementation and the overall fuzzy controller design methodology 1. Are there computational advantages in using fuzzy control as compared to conventional control? Not always. PID control is simpler than fuzzy control; however, there are some types of conventional control that are very difficult to implement where a fuzzy controller can be simpler. It depends on the application and the methods you choose 2. Should I use a conventional or"fuzzy processor"for implementation? We have typically found that our needs can be met if we use a conventional processor that has a better track record with reliability; however, there may be some advantages to fuzzy processors when large rule-bases are used and fast sampling times are nee 3. Are there special"tricks of the trade"in the implementation of fuzzy controllers that have many rules? Yes 4. Does fuzzy control provide for a user-friendly way to tune the controller during implementation studies? Often it does. We have found in field studies that when you know generally what to do to get a controller to work, it is sometimes hard to get this information into the gains of a conventional controller and easier to express it in rules and load them into a fuzzy con troller or fuzzy supervisor Overall. in ing fuzzy to conventional control, it is interesting to note that there are conventional control schemes that are analogous to fuzzy ones: (1)direct fuzzy control is analogous to direct nonlinear control, (2)fuzzy adaptive control is analogous to conventional adaptive control (e.g, model reference adaptive control), and (3) fuzzy supervisory control is analogous to hierarchical control. Does there exist an analogous conventional approach to every the performance of the fuzzy versus the convent ona approach it seems to be very important to compare and contrast 5.3 Neural Networks Artificial neural networks are circuits, computer algorithms, or mathematical representations of the massively connected set of neurons that form biological neural networks. They have been shown to be useful as an alternative computing echnology and have proven useful in a variety of pattern recognition, signal processing, estimation, and control problems. Their capabilities to learn from examples have been particularly useful In this section we will introduce two of the more popular neural networks and discuss how they relate to the areas of uzzy systems and control. We must emphasize that there are many topics in the area of neural networks that are not
Several of these approaches may apply to the analysis of the behavior of the fuzzy control system you design. 4. What are the problems with utilizing mathematical analysis for fuzzy control system verification? The techniques take time to learn. The problems for which fuzzy control are particularly well suited, and where there is often very good motivation to use fuzzy rather than conventional control, are the control problems where the plant has complex nonlinear behavior, and where a model is hard to derive due to inherent uncertainties. Each of these characteristics often makes the assumptions that are needed for the nonlinear analysis techniques invalid, so the theory often does not end up offering much when it is really needed. 5. Does fuzzy control provide "robust control"? If so, can this be demonstrated mathematically or experimentally? There has been a recent focus in research on stability analysis to show that fuzzy control does provide robust control. It is very difficult, of course, to show robustness via experimentation since by its very definition robustness verification requires extensive experimentation (e.g., you could not call the fuzzy controller for the rotational inverted pendulum case or "robust" when it was only shown to be successful for one disturbance condition). 5.2.3 Implementation and General Issues Finally, we will discuss several issues related to implementation and the overall fuzzy controller design methodology. 1. Are there computational advantages in using fuzzy control as compared to conventional control? Not always. PID control is simpler than fuzzy control; however, there are some types of conventional control that are very difficult to implement where a fuzzy controller can be simpler. It depends on the application and the methods you choose. 2. Should I use a conventional or "fuzzy processor" for implementation? We have typically found that our needs can be met if we use a conventional processor that has a better track record with reliability; however, there may be some advantages to fuzzy processors when large rule-bases are used and fast sampling times are needed. 3. Are there special "tricks of the trade" in the implementation of fuzzy controllers that have many rules? Yes. 4. Does fuzzy control provide for a user-friendly way to tune the controller during implementation studies? Often it does. We have found in field studies that when you know generally what to do to get a controller to work, it is sometimes hard to get this information into the gains of a conventional controller and easier to express it in rules and load them into a fuzzy controller or fuzzy supervisor. Overall, in comparing fuzzy to conventional control, it is interesting to note that there are conventional control schemes that are analogous to fuzzy ones: (1) direct fuzzy control is analogous to direct nonlinear control, (2) fuzzy adaptive control is analogous to conventional adaptive control (e.g., model reference adaptive control), and (3) fuzzy supervisory control is analogous to hierarchical control. Does there exist an analogous conventional approach to every fuzzy control scheme? If so, then in doing fuzzy control research it seems to be very important to compare and contrast the performance of the fuzzy versus the conventional approaches. 5.3 Neural Networks Artificial neural networks are circuits, computer algorithms, or mathematical representations of the massively connected set of neurons that form biological neural networks. They have been shown to be useful as an alternative computing technology and have proven useful in a variety of pattern recognition, signal processing, estimation, and control problems. Their capabilities to learn from examples have been particularly useful. In this section we will introduce two of the more popular neural networks and discuss how they relate to the areas of fuzzy systems and control. We must emphasize that there are many topics in the area of neural networks that are not
covered here. For instance, we do not discuss associative memories and Hopfield neural networks, recurrent networks Boltz-mann machines, or Hebbian or competitive learning. We refer the reader to Section 5.8, For Further Study, for references that cover these topics in detail 5.3.1 Multilayer Perceptrons The multilayer perceptron is a feed-forward neural network (i.e, it does not use past values of its outputs or other internal variables to compute its current output). It is composed of an interconnection of basic neuron processing units The Neuron For a single neuron, suppose that we use, i=1, 2, , n, to denote its inputs and suppose that it has a single output y Figure 5. 1 shows the neuron. Such a neuron first forms a weighted sum of the inputs 2nx|- wherea are the interconnection"weights"andes the"bias"for the neuron(these parameters model the interconnections between the cell bodies in the neurons of a biological neural network ). The signal represents a signal in the biological neuron, and the processing that the neuron performs on this signal is represented with an"activation function. This activation function is represented with a function f, and the output that it computes y=f(-)=川mx-0 (5.1) Basically, the neuron model represents the biological neuron that"fires"(turns on) when its inputs are significantly excited (ie, is big enough). The manner in which the neuron fires is defined by the activation function f. There are many ways to define the activation function Threshold function: For this type of activation function we have if f( 0 if 2<0 so that once the input signal is above zero the neuron turns on a Sigmoid function: For this type of activation function we have f(=)=1+eXp(-bx) so that the input signal continuously turns on the neuron an increasing amount as it increases(plot the function values against z to convince yourself of this). The parameter b affects the slope of the sigmoid function. There are many functions that take on a shape that is sigmoidal. For instance, one that is often used in neural network is the hyperbolic tangent function f(a)=tanh(-) 1-exp(=) 21+exp() Equation(5.1), with one of the above activation functions, represents the computations made by one neuron in the neural network. Next, we define how we interconnect these neurons to form a neural network-in particular, the multilayer perceptron
covered here. For instance, we do not discuss associative memories and Hopfield neural networks, recurrent networks, Boltz-mann machines, or Hebbian or competitive learning. We refer the reader to Section 5.8, For Further Study, for references that cover these topics in detail. 5.3.1 Multilayer Perceptrons The multilayer perceptron is a feed-forward neural network (i.e., it does not use past values of its outputs or other internal variables to compute its current output). It is composed of an interconnection of basic neuron processing units. The Neuron For a single neuron, suppose that we use , i = 1,2,..., n, to denote its inputs and suppose that it has a single output y. Figure 5.1 shows the neuron. Such a neuron first forms a weighted sum of the inputs 1 n i i i z x ω θ = ⎛ ⎞ = − ⎜ ⎟ ⎝ ⎠ ∑ whereωi are the interconnection "weights" andθis the "bias" for the neuron (these parameters model the interconnections between the cell bodies in the neurons of a biological neural network). The signal z represents a signal in the biological neuron, and the processing that the neuron performs on this signal is represented with an "activation function." This activation function is represented with a function f, and the output that it computes is ( ) 1 n i i i y fz f x ω θ = ⎛ ⎞ ⎛ ⎞ == − ⎜⎜ ⎟ ⎝ ⎠ ⎝ ⎠ ∑ ⎟ ( 5.1) Basically, the neuron model represents the biological neuron that "fires" (turns on) when its inputs are significantly excited (i.e., z is big enough). The manner in which the neuron fires is defined by the activation function f . There are many ways to define the activation function: Threshold function: For this type of activation function we have ( ) 1 if 0 0 if 0 z f z z ⎧ ≥ = ⎨ ⎩ < so that once the input signal z is above zero the neuron turns on. Sigmoid function: For this type of activation function we have 1 ( ) 1 exp( ) f z bz = + − (5.2) so that the input signal z continuously turns on the neuron an increasing amount as it increases (plot the function values against z to convince yourself of this).The parameter b affects the slope of the sigmoid function. There are many functions that take on a shape that is sigmoidal. For instance, one that is often used in neural networks is the hyperbolic tangent function 1 exp( ) ( ) tanh( ) 2 1 exp( ) z z f z z − = = + Equation (5.1), with one of the above activation functions, represents the computations made by one neuron in the neural network. Next, we define how we interconnect these neurons to form a neural network—in particular, the multilayer perceptron
是it f(z Figure 5.1 Single neuron model Network of neurons The basic structure for the multilayer perceptron is shown in Figure 5.2. There, the circles represent the neurons(weights bias, and activation function) and the lines represent the connections between the inputs and neurons, and between the neurons in one layer and those in the next layer. This is a three-layer perceptron since there are three stages of neural processing between the inputs and outputs. More layers can be added by concatenating additional"hidden"layers of neurons The multilayer perceptron has inputs, i=1, 2,,n, and outputs, j=1, 2,, m. The number of neurons in the first hidden layer(see Figure 5.2)is. In the second hidden layer there are neurons, and in the output layer there are m neure Hence, in an N layer perceptron there are "i neurons in the / hidden layer, i=1,2,N-I First Shead Output aidian nEde lasee Figure 5.2 multilayer perceptron models The neurons in the first layer of the multilayer perceptron perform computations, and the outputs of these neurons are x=f(∑vg"x)-6 with=1,2.n/. The neurons in the second layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by x=°(∑x2)-02) withj=1,2,., nz. The neurons in the third layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by
Figure 5.1 Single neuron model Network of Neurons The basic structure for the multilayer perceptron is shown in Figure 5.2. There, the circles represent the neurons (weights, bias, and activation function) and the lines represent the connections between the inputs and neurons, and between the neurons in one layer and those in the next layer. This is a three-layer perceptron since there are three stages of neural processing between the inputs and outputs. More layers can be added by concatenating additional "hidden" layers of neurons. The multilayer perceptron has inputs, i = 1,2,..., n, and outputs , j =1,2,..., m. The number of neurons in the first hidden layer (see Figure 5.2) is . In the second hidden layer there are neurons, and in the output layer there are m neurons. Hence, in an N layer perceptron there are i neurons in the i n th hidden layer, i = 1,2,..., N- 1. Figure 5.2 multilayer perceptron models. The neurons in the first layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by (1) (1) (1) (1) 1 (( ) ) n j j ij i j i x f wx θ = = − ∑ with j = 1,2.....n1. The neurons in the second layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by 1 (2) (2) (2) (1) (2) 1 (( ) ) n j j ij i j i x f wx θ = = − ∑ with j = 1,2,..., n2. The neurons in the third layer of the multilayer perceptron perform computations, and the outputs of these neurons are given by
y=/(∑x2) with i=1, 2,...,m The parameters(scalar real numbers)w) are called the weights of the first hidden layer. The w@2)are called the weights of the second hidden layer. The w are called the weights of the output layer. The parameters 0, are called the biases of the first hidden layer. The parameters 0(are called the biases of the second hidden layer, and the 0, are the biases of the output layer. The functions f (for the output layer), f (2)(for the second hidden layer), and f (for the first hidden layer) represent the activation functions. The activation functions can be different for each neuron in the multilayer perception(e.g, the first layer could have one type of sigmoid, while the next two layers could have different sigmo id functions or threshold functions) This completes the definition of the multilayer perception. Next, we will introduce the radial basis function neural network. After that we explain how both of these neural networks relate to the other topics covered in this book 5.3.2 Radial Basis Function Neural Networks A locally tuned, overlapping receptive field is found in parts of the cerebral cortex, in the visual cortex, and in other parts of the brain. The radial basis function neural network model is based on these biological systems A radial basis function neural network is shown in Figure 5.3. There, the inputs are xi, i=1, 2,, n, and the output is ,f(x)where f represents the processing by the entire radial basis function neural network. Letx=[-1, r2 x, .The input to the i receptive field unit is x, and its output is denoted with R;(x). It has what is called a"strength"which denote by y,. Assume that there are M receptive field units. Hence, from Figure 5.3 y=f(x)=∑R(x) is the output of the radial basis function neural network Figure 5.3 Radial basis function neural network model There are several possible choices for the"receptive field units"Ri(x) 'e could choose R()=ex Wheres=[ci c.c]. o, is a scalar, and if: is a vector thenI=I=V==
2 (2) 1 (( ) ) n j j ij i j i y f wx θ = = − ∑ with j = 1,2, ...,m. The parameters (scalar real numbers) (1) ij w are called the weights of the first hidden layer. The (2) ij w are called the weights of the second hidden layer. The are called the weights of the output layer. The parameters wij (1) θ j are called the biases of the first hidden layer. The parameters (2) θ j are called the biases of the second hidden layer, and theθ j are the biases of the output layer. The functions fj (for the output layer), (2) j f (for the second hidden layer), and (1) j f (for the first hidden layer) represent the activation functions. The activation functions can be different for each neuron in the multilayer perception (e.g., the first layer could have one type of sigmoid, while the next two layers could have different sigmoid functions or threshold functions). This completes the definition of the multilayer perception. Next, we will introduce the radial basis function neural network. After that we explain how both of these neural networks relate to the other topics covered in this book. 5.3.2 Radial Basis Function Neural Networks A locally tuned, overlapping receptive field is found in parts of the cerebral cortex, in the visual cortex, and in other parts of the brain. The radial basis function neural network model is based on these biological systems. A radial basis function neural network is shown in Figure 5.3. There, the inputs are xi, i = 1,2,..., n, and the output is y =f(x) where f represents the processing by the entire radial basis function neural network. Let [ 1 2 , , ] T n x = xx x " . The input to the i th receptive field unit is x, and its output is denoted with Ri (x). lt has what is called a "strength" which we denote by i y . Assume that there are M receptive field units. Hence, from Figure 5.3, ( ) ( ) 1 M i i i y f x yR x = = = ∑ (5.3) is the output of the radial basis function neural network. Figure 5.3 Radial basis function neural network model. There are several possible choices for the "receptive field units" R i (x): 1. We could choose ( ) 2 2 exp i i i x c R x σ ⎛ ⎞ − = ⎜− ⎟ ⎜ ⎟ ⎝ ⎠ Where 1 2 , , T ii i i n ⎤ ⎦ c cc c = ⎡ ⎣ … i ,σ is a scalar, and if z is a vector then T z = z z
2. We could choose R(x)= where c. and o. are defined in choice 1 There are also alternatives to how to compute the output of the radial basis function neural network. For instance rather than computing the simple sum as in Equation(5.3), you could compute a weighted average . R( y=f(x)=iel (54) ∑R(x) It is also possible to define multilayer radial basis function neural networks This completes the definition of the radial basis function neural network. Next, we explain the relationshil between multilayer perceptions and radial basis function neural networks and fuzzy systems 5.3.3 Relationships Between Fuzzy Systems and Neural Networks There are two ways in which there are relationships between fuzzy systems and neural networks. First, techniques from one area can be used in the other. Second, in some cases the functionality (i.e, the nonlinear function that they implement) is identical. Some label the intersection between fuzzy systems and neural networks with the term fuzzy-neural"or"neuro-fuzzy"to highlight that techniques from both fields are being used. Here, we avoid this terminology and simply highlight the basic relationships between the two field Multilayer Perceptrons The multilayer perceptron should be viewed as a nonlinear network whose nonlinearity can be tuned by changing the weights, biases, and parameters of the activation functions. The fuzzy system is also a tunable nonlinearity whose shape can be changed by tuning, for example, the membership functions. Since both are tunable nonlinearities, the following approaches are possible a Gradient methods can be used for training neural networks to perform system identification or to act as estimators or predictors in the same way as fuzzy systems were trained Indeed, the gradient training of neural networks, called"back-propagation training, "was introduced well before the gradient training of fuzzy systems, and the idea for training fuzzy systems this way came from the field of neural networks Hybrid methods for training can also be used for neural networks. For instance, gradient methods may be used in conjunction with clustering methods applied to neural networks a Indirect adaptive control can also be achieved with a multilayer perceptron. To do this we use two multilayer perceptrons as the tunable nonlinearities in the certainty equivalence control law and the gradient method for tuning a Gain scheduled control may be achieved by training a multilayer perceptron to map the associations between operating conditions and controller parameters This list is by no means exhaustive. It simply shows that multilayer pere networks can take on a similar role to that of a fuzzy system in performing the function of being a tunable nonlinearity. An advantage that the fuzzy system may have, however, is that it often facilitates the incorporation of heuristic knowledge into the solution to the problem, which
2. We could choose ( ) 2 2 1 1 exp i i i R x x c σ = ⎛ ⎞ − + −⎜ ⎟ ⎜ ⎟ ⎝ ⎠ where i c and σ i are defined in choice 1. There are also alternatives to how to compute the output of the radial basis function neural network. For instance, rather than computing the simple sum as in Equation (5.3), you could compute a weighted average ( ) ( ) ( ) 1 1 M i i i M i i y R x y fx R x = = = = ∑ ∑ （5.4） It is also possible to define multilayer radial basis function neural networks. This completes the definition of the radial basis function neural network. Next, we explain the relationships between multilayer perceptions and radial basis function neural networks and fuzzy systems. 5.3.3 Relationships Between Fuzzy Systems and Neural Networks There are two ways in which there are relationships between fuzzy systems and neural networks. First, techniques from one area can be used in the other. Second, in some cases the functionality (i.e., the nonlinear function that they implement) is identical. Some label the intersection between fuzzy systems and neural networks with the term "fuzzy-neural" or "neuro-fuzzy" to highlight that techniques from both fields are being used. Here, we avoid this terminology and simply highlight the basic relationships between the two fields. Multilayer Perceptrons The multilayer perceptron should be viewed as a nonlinear network whose nonlinearity can be tuned by changing the weights, biases, and parameters of the activation functions. The fuzzy system is also a tunable nonlinearity whose shape can be changed by tuning, for example, the membership functions. Since both are tunable nonlinearities, the following approaches are possible: Gradient methods can be used for training neural networks to perform system identification or to act as estimators or predictors in the same way as fuzzy systems were trained. Indeed, the gradient training of neural networks, called "back-propagation training," was introduced well before the gradient training of fuzzy systems, and the idea for training fuzzy systems this way came from the field of neural networks. Hybrid methods for training can also be used for neural networks. For instance, gradient methods may be used in conjunction with clustering methods applied to neural networks. Indirect adaptive control can also be achieved with a multilayer perceptron. To do this we use two multilayer perceptrons as the tunable nonlinearities in the certainty equivalence control law and the gradient method for tuning. Gain scheduled control may be achieved by training a multilayer perceptron to map the associations between operating conditions and controller parameters. This list is by no means exhaustive. It simply shows that multilayer perceptron networks can take on a similar role to that of a fuzzy system in performing the function of being a tunable nonlinearity. An advantage that the fuzzy system may have, however, is that it often facilitates the incorporation of heuristic knowledge into the solution to the problem, which
can,at times, have a significant impact on the quality of the solutio Radial basis Function Neural Networks Some radial basis function neural networks are equivalent to some standard fuzzy systems in the sense that they are functionally equivalent (ie, given the same inputs, they will produce the same outputs). To see this, suppose that in Equation(5.4)we let M=R(i.e, the number of receptive field units equal to the number of rules ), y,=b, (i.., the receptive field unit strengths equal to the output membership function centers ), and choose the receptive field units R(x)=(x) (.e, choose the receptive field units to be the same as the premise membership functions). In this case we see that the radial basis function neural network is identical to a certain fuzzy system that uses center-average defuzzification. This fuzzy system is then given by y=f(x) bu (x) (x) It is also interesting to note that the functional fuzzy system(the more general version of the Takagi-Sugeno fuzzy stem)is equivalent to a class of two-layer neural networks (2001 The equivalence between this type of fuzzy system and a radial basis function neural network shows that all the echniques in this book for the above type of fuzzy system work in the same way for the above type of radial basis function neural network (or, using [200, the techniques for the Takagi fuzzy system can be used for a type of multilayer radial basis function neural network) Due to the above relationships between fuzzy systems and neural networks, some would like to view fuzzy sys d neural networks as identical areas This is however not the case for the following reasons There are classes of neural networks(e.g, dynamic neural networks) that may have a fuzzy system analog, but if so it would have to include not only standard fuzzy components but some form of a differential equation component There are certain fuzzy systems that have no clear neural analog. Consider, for example, certain"fuzzy dynamic systems".We can, however, envision how you could go about "designing a neural analog to such fuzzy systems using gradient methods like back-propagation)using data, often without using extra heuristic knowledge w? ip The neural network has traditionally been a"black box"approach where the weights and biases are trained(e often have. In fuzzy systems you can incorporate heuristic information and use data to train them. This last difference is often quoted as being one of the advantages of fuzzy systems over neural networks, at least for some applications Regardless of the differences, it is important to note that many methods in neural control (i.e, when we use a neural network for the control of a system) are quite similar to those in adaptive fuzzy control. For instance, since the fuzzy system and radial basis function neural network can be linearly parameterized, we can use them as the identifier structures in direct or indirect adaptive control schemes and use gradient or least squares methods to update the parameters. Indeed, we could have used neural networks as the structure that we trained for all of the identification methods. In this sense we can use neural networks in system identification, estimation, and prediction, and as a direct (fixed) controller that is trained with input-output data. Basically, to be fluent with the methods of adaptive fuzzy systems and control, you must know the methods of neural control-and vice versa
can, at times, have a significant impact on the quality of the solution. Radial Basis Function Neural Networks Some radial basis function neural networks are equivalent to some standard fuzzy systems in the sense that they are functionally equivalent (i.e., given the same inputs, they will produce the same outputs). To see this, suppose that in Equation (5.4) we let M =R (i.e., the number of receptive field units equal to the number of rules), i y b = i (i.e., the receptive field unit strengths equal to the output membership function centers), and choose the receptive field units as () () Ri i x x = μ (i.e., choose the receptive field units to be the same as the premise membership functions). In this case we see that the radial basis function neural network is identical to a certain fuzzy system that uses center-average defuzzification. This fuzzy system is then given by 1 1 ( ) ( ) ( ) R i i i R i i b x y fx x μ μ = = = = ∑ ∑ It is also interesting to note that the functional fuzzy system (the more general version of the Takagi-Sugeno fuzzy system) is equivalent to a class of two-layer neural networks [200]. The equivalence between this type of fuzzy system and a radial basis function neural network shows that all the techniques in this book for the above type of fuzzy system work in the same way for the above type of radial basis function neural network (or, using [200], the techniques for the Takagi-Sugeno fuzzy system can be used for a type of multilayer radial basis function neural network). Due to the above relationships between fuzzy systems and neural networks, some would like to view fuzzy systems and neural networks as identical areas. This is, however, not the case for the following reasons: There are classes of neural networks (e.g., dynamic neural networks) that may have a fuzzy system analog, but if so it would have to include not only standard fuzzy components but some form of a differential equation component. There are certain fuzzy systems that have no clear neural analog. Consider, for example, certain "fuzzy dynamic systems". We can, however, envision how you could go about "designing a neural analog to such fuzzy systems. The neural network has traditionally been a "black box" approach where the weights and biases are trained (e.g., using gradient methods like back-propagation) using data, often without using extra heuristic knowledge we often have. In fuzzy systems you can incorporate heuristic information and use data to train them. This last difference is often quoted as being one of the advantages of fuzzy systems over neural networks, at least for some applications. Regardless of the differences, it is important to note that many methods in neural control (i.e., when we use a neural network for the control of a system) are quite similar to those in adaptive fuzzy control. For instance, since the fuzzy system and radial basis function neural network can be linearly parameterized, we can use them as the identifier structures in direct or indirect adaptive control schemes and use gradient or least squares methods to update the parameters. Indeed, we could have used neural networks as the structure that we trained for all of the identification methods. In this sense we can use neural networks in system identification, estimation, and prediction, and as a direct (fixed) controller that is trained with input-output data. Basically, to be fluent with the methods of adaptive fuzzy systems and control, you must know the methods of neural control—and vice versa
5.4 Genetic Algorithms A genetic algorithm(GA)uses the principles of evolution, natural selection, and genetics from natural biological systems in a computer algorithm to simulate evolution. Essentially, the genetic algorithm is an optimization technique that performs a parallel, stochastic, but directed search to evolve the most fit population. In this section we will introduce the genetic algorithm and explain how it can be used for design and tuning of fuzzy systems 5.4.1 Genetic Algorithms: A Tutorial The genetic algorithm borrows ideas from and attempts to simulate Darwin,'s theory on natural selection and Mendel work in genetics on inheritance. The genetic algorithm is ah optimization technique that evaluates more than one area of the search space and can discover more than one solution to a problem. In particular, it provides a stochastic optimization method where if it"gets stuck"at a local optimum, it tries to simultaneously find other parts of the search space and jump out"of the local optimum to a global one Representation and the population of Individuals The"fitness function"measures the fitness of an individual to survive in a population of individuals. The genetic algorithm will seek to maximize the fitness function e) by selecting the individuals that we represent with 0. To represent the genetic algorithm in a computer, we make 8 a string. In particular, we show such a string in Figure 5.4.A string is a chromosome in a biological system. It is a string of"genes" that can take on different"alleles. In a computer we use number systems to encode alleles. Hence, a gene is a" digit location"that can take on different values from a number system(i.e, different types of alleles) Values here=alleles Values here=alleles String of genes=chromosome String of genes=chromosome Gene=digit location Gene=digit location Figure 5. 4 String for representing an individual For instance, in a base-2 number system, alleles come from the set 10, 13, while in a base-10 number system, allel come from the set(0, 1, 2,3,4, 5, 6, 7, 8, 9). Hence, abinary chromosome has zeros or ones in its gene locations. A Sp example, consider the binary chromosome 1011110001010, which is a binary string of length 13. If we are seeking to optimize parameters of a system that come in a base-10 number system then we will need to encode the numbers into the binary number system(using the standard method for the conversion of base-10 numbers to base-2 numbers). We will aso need to decode the binary strings into base-10 numbers to use them. Here, we will develop the genetic algorithm for base-2 or base-10 number systems but we will favor the use of the base-10 representation since then there is no need for encoding or decoding(which can be computationally expensive for on-line, real-time applications) As an example of a base-10 chromosome, consider 8219345127066 that has 13 gene positions. For such
5.4 Genetic Algorithms A genetic algorithm (GA) uses the principles of evolution, natural selection, and genetics from natural biological systems in a computer algorithm to simulate evolution. Essentially, the genetic algorithm is an optimization technique that performs a parallel, stochastic, but directed search to evolve the most fit population. In this section we will introduce the genetic algorithm and explain how it can be used for design and tuning of fuzzy systems. 5.4.1 Genetic Algorithms: A Tutorial The genetic algorithm borrows ideas from and attempts to simulate Darwin's theory on natural selection and Mendel's work in genetics on inheritance. The genetic algorithm is ah optimization technique that evaluates more than one area of the search space and can discover more than one solution to a problem. In particular, it provides a stochastic optimization method where if it "gets stuck" at a local optimum, it tries to simultaneously find other parts of the search space and "jump out" of the local optimum to a global one. Representation and the Population of Individuals The "fitness function" measures the fitness of an individual to survive in a population of individuals. The genetic algorithm will seek to maximize the fitness function J(θ) by selecting the individuals that we represent with θ . To represent the genetic algorithm in a computer, we make θ a string. In particular, we show such a string in Figure 5.4. A string is a chromosome in a biological system. It is a string of "genes" that can take on different "alleles." In a computer we use number systems to encode alleles. Hence, a gene is a "digit location" that can take on different values from a number system (i.e., different types of alleles). Values here=alleles Gene=digit location String of genes=chromosome Figure 5.4 String for representing an individual For instance, in a base-2 number system, alleles come from the set {0,1}, while in a base-l0 number system, alleles come from the set {0,1,2,3,4,5,6,7,,8,9}. Hence, abinary chromosome has zeros or ones in its gene locations. As an example, consider the binary chromosome 1011110001010, which is a binary string of length 13. If we are seeking to optimize parameters of a system that come in a base-10 number system then we will need to encode the numbers into the binary number system (using the standard method for the conversion of base-10 numbers to base-2 numbers). We will also need to decode the binary strings into base-10 numbers to use them. Here, we will develop the genetic algorithm for base-2 or base-10 number systems but we will favor the use of the base-10 representation since then there is no need for encoding or decoding (which can be computationally expensive for on-line, real-time applications). As an example of a base-10 chromosome, consider 8219345127066 that has 13 gene positions. For such