Computational Intelligence（Concepts to Implementations）Part 1

Preface Several computational analytic tools have matured in the last 10 to 15 years that facilitate solving problems that were previously difficult or impossible to solve.These new analytical tools,known collectively as computational intelligence tools,include artificial neural networks,fuzzy systems,and evolutionary computation.They have recently been combined among themselves as well as with more traditional approa- ches,such as statistical analysis,to solve extremely challenging problems.Diagnos- tic systems,for example,are being developed that include Bayesian,neural network, and rule-based diagnostic modules,evolutionary algorithm-based explanation facil- ities,and expert system shells.All of these components work together in a"seamless" way that is transparent to the user,and they deliver results that significantly exceed what is available with any single approach. At a system prototype level,computational intelligence(CI)tools are capable of yielding results in a relatively short time.For instance,the implementation of a conventional expert system often takes one to three years and requires the active participation of a "knowledge engineer"to build the knowledge and rule bases. In contrast,computational intelligence system solutions can often be prototyped in a few weeks to a few months and are implemented using available engineering and computational resources.Indeed,computational intelligence tools are capable of being applied in many instances by "domain experts"rather than solely by “computer gurus.” This means that biomedical engineers,for example,can solve problems in biomedical engineering without relying on outside computer science expertise such as that required to build knowledge bases for classical expert systems.Furthermore, innovative ways to combine CI tools are cropping up every day.For example,tools have been developed that incorporate knowledge elements with neural networks, fuzzy logic,and evolutionary computing theory.Such tools are able to solve quickly classification and clustering problems that would be extremely time consuming using other techniques. The concepts,paradigms,algorithms,and implementation of computational intelligence and its constituent methodologies-evolutionary computation,neural networks,and fuzzy logic-are the focus of this book.In addition,we emphasize practical applications throughout,that is,how to apply the concepts,paradigms, algorithms,and implementations discussed to practical problems in engi- neering and computer science.This emphasis culminates in the real-world case xiii

Preface Several computational analytic tools have matured in the last 10 to 15 years that facilitate solving problems that were previously difficult or impossible to solve. These new analytical tools, known collectively as computational intelligence tools, include artificial neural networks, fuzzy systems, and evolutionary computation. They have recently been combined among themselves as well as with more traditional approa￾ches, such as statistical analysis, to solve extremely challenging problems. Diagnos￾tic systems, for example, are being developed that include Bayesian, neural network, and rule-based diagnostic modules, evolutionary algorithm-based explanation facil￾ities, and expert system shells. All of these components work together in a "seamless" way that is transparent to the user, and they deliver results that significantly exceed what is available with any single approach. At a system prototype level, computational intelligence (CI) tools are capable of yielding results in a relatively short time. For instance, the implementation of a conventional expert system often takes one to three years and requires the active participation of a "knowledge engineer" to build the knowledge and rule bases. In contrast, computational intelligence system solutions can often be prototyped in a few weeks to a few months and are implemented using available engineering and computational resources. Indeed, computational intelligence tools are capable of being applied in many instances by "domain experts" rather than solely by "computer gurus." This means that biomedical engineers, for example, can solve problems in biomedical engineering without relying on outside computer science expertise such as that required to build knowledge bases for classical expert systems. Furthermore, innovative ways to combine CI tools are cropping up every day. For example, tools have been developed that incorporate knowledge elements with neural networks, fuzzy logic, and evolutionary computing theory. Such tools are able to solve quickly classification and clustering problems that would be extremely time consuming using other techniques. The concepts, paradigms, algorithms, and implementation of computational intelligence and its constituent methodologies~evolutionary computation, neural networks, and fuzzy logic~are the focus of this book. In addition, we emphasize practical applications throughout, that is, how to apply the concepts, paradigms, algorithms, and implementations discussed to practical problems in engi￾neering and computer science. This emphasis culminates in the real-world case imm XIII

Preface studies in a final chapter,which are available on this book's web site at http://www.computelligence.org/issue/CICI/CICI.html. Computational intelligence is closely related to the field called "soft computing." There is,in fact,a significant overlap.According to Lotfi Zadeh(1998),the inventor of fuzzy logic and one of the leading proponents of soft computing: Soft computing is not a single methodology.Rather,it is a consortium of computing methodologies which collectively provide a foundation for the conception,design and deployment of intelligent systems.At this juncture,the principal members ofsoft computing are fuzzy logic(FL),neurocomputing(NC),genetic computing (GC), and probabilistic computing(PC),with the last subsuming evidential reasoning, belief networks,chaotic systems,and parts of machine learning theory.In contrast to traditional hard computing,soft computing is tolerant of imprecision,uncertainty and partial truth.The guiding principle of soft computing is:"exploit the tolerance for imprecision,uncertainty and partial truth to achieve tractability,robustness,low solution cost and better rapport with reality." Zadeh also believes that soft computing is serving as the foundation for the emerg- ing field of computational intelligence,and that"In this perspective,the difference between traditional AI [artificial intelligence]and computational intelligence is that AI is based on hard computing whereas CI is based on soft computing"(Zadeh 1994).We believe that soft computing is a large subset of computational intelligence. We heartily agree with him when he says,"Hybrid intelligent systems are definitely the wave of the future"(Zadeh 1994). Some of the material in this book is adapted from Computational Intelligence PC Tools by Eberhart,Dobbins,and Simpson(Academic Press 1996).The extensive rewrite and reorganization of that material reflect the change in our perception of computational intelligence that has occurred over the years.That change is reflected in an increased emphasis on evolutionary computation as providing a foundation for CI.It also features significant recent developments in particle swarm optimiza- tion and other evolutionary computation tools. The primary intended audience for Computational Intelligence:Concepts to Implementations comprises researchers and graduate students with engineering or computer science backgrounds and those with a special interest in computational intelligence and/or system adaptation.One of the authors [RE]has taught a CI intro- ductory course for several years;the material in this book was developed to support that course.Other audiences include researchers in fields such as cognitive science and the physical sciences and those who are motivated to learn about computational intelligence via self-study.We assume this book's users understand the basic con- cepts of classical(two-valued)logic,classical set theory,and elementary probability theory.We also assume that readers have a familiarity with computers and a very basic familiarity with calculus.Knowledge of a computer language such as Java,C, or Visual BASIC is very helpful but not required

Preface studies in a final chapter, which are available on this book's web site at hap://www.computelligence.org/issue/CICI/CICI, html. Computational intelligence is closely related to the field called "soft computing." There is, in fact, a significant overlap. According to Lotfi Zadeh (1998), the inventor of fuzzy logic and one of the leading proponents of soft computing: Soft computing is not a single methodology. Rather, it is a consortium of computing methodologies which collectively provide a foundation for the conception, design and deployment of intelligent systems. At this juncture, the principal members of soft computing are fuzzy logic (FL), neurocomputing (NC), genetic computing (GC), and probabilistic computing (PC), with the last subsuming evidential reasoning, belief networks, chaotic systems, and parts of machine learning theory. In contrast to traditional hard computing, soft computing is tolerant of imprecision, uncertainty and partial truth. The guiding principle of soft computing is: "exploit the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness, low solution cost and better rapport with reality." Zadeh also believes that soft computing is serving as the foundation for the emerg￾ing field of computational intelligence, and that "In this perspective, the difference between traditional AI [artificial intelligence] and computational intelligence is that AI is based on hard computing whereas CI is based on soft computing" (Zadeh 1994). We believe that soft computing is a large subset of computational intelligence. We heartily agree with him when he says, "Hybrid intelligent systems are definitely the wave of the future" (Zadeh 1994). Some of the material in this book is adapted from Computational Intelligence PC Tools by Eberhart, Dobbins, and Simpson (Academic Press 1996). The extensive rewrite and reorganization of that material reflect the change in our perception of computational intelligence that has occurred over the years. That change is reflected in an increased emphasis on evolutionary computation as providing a foundation for CI. It also features significant recent developments in particle swarm optimiza￾tion and other evolutionary computation tools. The primary intended audience for Computational Intelligence: Concepts to Implementations comprises researchers and graduate students with engineering or computer science backgrounds and those with a special interest in computational intelligence and/or system adaptation. One ofthe authors [RE] has taught a CI intro￾ductory course for several years; the material in this book was developed to support that course. Other audiences include researchers in fields such as cognitive science and the physical sciences and those who are motivated to learn about computational intelligence via self-study. We assume this book's users understand the basic con￾cepts of classical (two-valued) logic, classical set theory, and elementary probability theory. We also assume that readers have a familiarity with computers and a very basic familiarity with calculus. Knowledge of a computer language such as Java, C, or Visual BASIC is very helpful but not required

Preface The implementation chapters frequently refer to and list portions of computer code.In Chapters 4 and 6 we use the most common general-purpose,procedural programming language,C,to implement the evolutionary algorithms and the arti- ficial neural networks.Data structures,routines,and finite state machines are used extensively in the C programming.In Chapters 8 and 9,reflecting programming lan- guage evolution trends,we use an object-oriented programming language instead of the procedural programming language C to implement the fuzzy systems and evolu- tionary fuzzy systems.There are a variety of object-oriented languages,such as C++, Java,and C#.We use C++here primarily because it can be looked at as an extension of the C language. Organization of the Book This book is divided into twelve chapters.Chapters 1 and 2 lay the groundwork for the topic,introducing computational intelligence and its foundations.The next por- tion of the book includes the "backbone"chapters on the three main constituents of CI:evolutionary computation,neural networks,and fuzzy logic,in that order.This order provides an initial focus on evolutionary computation,which is presented as providing a foundation for development of computational intelligence tools involv- ing neural networks and fuzzy logic.For instance,when we discuss neural networks, we see how evolutionary computation can be used to evolve the weights and struc- ture of feedforward neural networks,and with fuzzy logic,we examine evolutionary computation applications to tools built using fuzzy logic.In other words,the evo- lutionary computation theme pervades this book.Within each backbone chapter, we discuss the histories of computational intelligence,evolutionary computation, neural networks,and fuzzy logic. We follow each backbone chapter with a chapter discussing implementation and examples.Each one contains a section on implementation considerations that addresses features frequently incorporated into these implementations,which fea- tures we chose and why we chose them,and the guidelines to using them,as well as interactions among them.The implementation chapters are intended to provide readers with the insight to clearly understand "canned,"commercially packaged software applications and to enable a more thorough understanding of software and hardware implementation issues for CI paradigms. Each chapter ends with exercises. Chapters'Contents Chapter 1,Foundations,defines terms used throughout the book and briefly reviews biological and behavioral motivations for the constituent methodologies of compu- tational intelligence.This is followed by a brief review of the major application areas

Preface ~."~ The implementation chapters frequently refer to and list portions of computer code. In Chapters 4 and 6 we use the most common general-purpose, procedural programming language, C, to implement the evolutionary algorithms and the arti￾ficial neural networks. Data structures, routines, and finite state machines are used extensively in the C programming. In Chapters 8 and 9, reflecting programming lan￾guage evolution trends, we use an object-oriented programming language instead of the procedural programming language C to implement the fuzzy systems and evolu￾tionary fuzzy systems. There are a variety of object-oriented languages, such as C++, Java, and C#. We use C++ here primarily because it can be looked at as an extension of the C language. Organization of the Book This book is divided into twelve chapters. Chapters 1 and 2 lay the groundwork for the topic, introducing computational intelligence and its foundations. The next por￾tion of the book includes the "backbone" chapters on the three main constituents of CI: evolutionary computation, neural networks, and fuzzy logic, in that order. This order provides an initial focus on evolutionary computation, which is presented as providing a foundation for development of computational intelligence tools involv￾ing neural networks and fuzzy logic. For instance, when we discuss neural networks, we see how evolutionary computation can be used to evolve the weights and struc￾ture of feedforward neural networks, and with fuzzy logic, we examine evolutionary computation applications to tools built using fuzzy logic. In other words, the evo￾lutionary computation theme pervades this book. Within each backbone chapter, we discuss the histories of computational intelligence, evolutionary computation, neural networks, and fuzzy logic. We follow each backbone chapter with a chapter discussing implementation and examples. Each one contains a section on implementation considerations that addresses features frequently incorporated into these implementations, which fea￾tures we chose and why we chose them, and the guidelines to using them, as well as interactions among them. The implementation chapters are intended to provide readers with the insight to clearly understand "canned," commercially packaged software applications and to enable a more thorough understanding of software and hardware implementation issues for CI paradigms. Each chapter ends with exercises. Chapters" Contents Chapter 1, Foundations, defines terms used throughout the book and briefly reviews biological and behavioral motivations for the constituent methodologies of compu￾tational intelligence. This is followed by a brief review of the major application areas

Preface for each methodology,as well as of CI.The chapter concludes with a review of major computational intelligence application areas. Chapter 2,Computational Intelligence,launches directly into the core subject of this book.We first review the concepts of adaptation and self-organization,key to our view of computational intelligence.Then we summarize the brief history of the CI field,viewing it from the perspectives of other researchers.This leads us into a discussion of the relationships among the three major components and how they cooperate and/or are integrated into a computational intelligence system.We present our definition of computational intelligence,supported by diagrams that place it into context. Chapter 3,Evolutionary Computation:Concepts and Paradigms,has been adapted from the Evolutionary Computation Theory and Paradigms chapter in Swarm Intelligence(Kennedy,Eberhart,and Shi 2001)with updates and augmen- tations,including recent developments in particle swarm optimization and other evolutionary computation approaches.After reviewing the history of evolutionary computation and giving an overview of the field,we discuss its main paradigms: genetic algorithms,evolutionary programming,evolution strategies,genetic pro- gramming,and particle swarm optimization. Chapter 4,Evolutionary Computation Implementations,discusses factors to con- sider when implementing evolutionary computation paradigms and presents two implementation examples:a canonical genetic algorithm and a real-valued particle swarm that can be run in single-swarm or multiswarm configurations. Chapter 5,Neural Network Concepts and Paradigms,first briefly presents an overview of the history of neural networks,then examines what they are and why they are useful.A discussion of neural network components and terminology fol- lows,with a review of neural network topologies.A more detailed look at neural network learning and recall comes next,focusing on three of the most common neu- ral network paradigms:back-propagation,learning vector quantization,and self- organizing feature map networks.These networks represent the two basic learning types:supervised learning(back-propagation)and unsupervised learning(learning vector quantization and self-organizing feature maps).We also briefly discuss hybrid networks and recurrent networks.Finally,considerations of preprocessing and post- processing are evaluated. Chapter 6,Neural Network Implementations,discusses factors to consider when implementing artificial neural networks and presents four implementation exam- ples:back-propagation,learning vector quantization,self-organizing feature maps, and evolutionary neural networks. Chapter 7,Fuzzy Systems Concepts and Paradigms,leads off with a brief review of the history of the field,followed by an examination of fuzzy sets and fuzzy logic, the concepts of fuzzy sets,and approximate reasoning.We stress the differences between fuzzy logic and probability,and we present both Mamdani and Takagi- Sugeno-Kang approaches to the design and analysis of fuzzy systems.The chapter

Preface for each methodology, as well as of CI. The chapter concludes with a review of major computational intelligence application areas. Chapter 2, Computational Intelligence, launches directly into the core subject of this book. We first review the concepts of adaptation and self-organization, key to our view of computational intelligence. Then we summarize the brief history of the CI field, viewing it from the perspectives of other researchers. This leads us into a discussion of the relationships among the three major components and how they cooperate and/or are integrated into a computational intelligence system. We present our definition of computational intelligence, supported by diagrams that place it into context. Chapter 3, Evolutionary Computation: Concepts and Paradigms, has been adapted from the Evolutionary Computation Theory and Paradigms chapter in Swarm Intelligence (Kennedy, Eberhart, and Shi 2001) with updates and augmen￾tations, including recent developments in particle swarm optimization and other evolutionary computation approaches. After reviewing the history of evolutionary computation and giving an overview of the field, we discuss its main paradigms: genetic algorithms, evolutionary programming, evolution strategies, genetic pro￾gramming, and particle swarm optimization. Chapter 4, Evolutionary Computation Implementations, discusses factors to con￾sider when implementing evolutionary computation paradigms and presents two implementation examples: a canonical genetic algorithm and a real-valued particle swarm that can be run in single-swarm or multiswarm configurations. Chapter 5, Neural Network Concepts and Paradigms, first briefly presents an overview of the history of neural networks, then examines what they are and why they are useful. A discussion of neural network components and terminology fol￾lows, with a review of neural network topologies. A more detailed look at neural network learning and recall comes next, focusing on three of the most common neu￾ral network paradigms: back-propagation, learning vector quantization, and self￾organizing feature map networks. These networks represent the two basic learning types: supervised learning (back-propagation) and unsupervised learning (learning vector quantization and self-organizing feature maps). We also briefly discuss hybrid networks and recurrent networks. Finally, considerations ofpreprocessing and post￾processing are evaluated. Chapter 6, Neural Network Implementations, discusses factors to consider when implementing artificial neural networks and presents four implementation exam￾ples: back-propagation, learning vector quantization, self-organizing feature maps, and evolutionary neural networks. Chapter 7, Fuzzy Systems Concepts and Paradigms, leads off with a brief review of the history of the field, followed by an examination of fuzzy sets and fuzzy logic, the concepts of fuzzy sets, and approximate reasoning. We stress the differences between fuzzy logic and probability, and we present both Mamdani and Takagi￾Sugeno-Kang approaches to the design and analysis of fuzzy systems. The chapter

Preface concludes with a look at some design considerations and special topics related to fuzzy systems. Chapter 8,Fuzzy System Implementations,discusses factors to consider when implementing fuzzy systems and presents two implementation examples:a tradi- tional fuzzy rule system and an evolutionary fuzzy rule system.The evolutionary fuzzy rule system provides a transition into computational intelligence systems. Chapter 9,Computational Intelligence Implementations,reflects recent devel- opments in the field,including evolutionary fuzzy systems and approaches to sys- tem adaptation using computational intelligence.We expand the discussion of the interaction and cooperation among the three basic components of CI and include a section on adaptive evolutionary computation using fuzzy systems. Chapter 10,Performance Metrics,includes a number of system performance measures not generally used in other disciplines.Included are percent correct,sum- squared error,absolute error,normalized error,receiver operating characteristic curves,recall and precision,confusion matrices,and the chi-squared test. Chapter 11,Analysis and Explanation,presents several tools that are helpful in assessing and explaining how well a computational intelligence tool is doing its job. Included are sensitivity analyses,Hinton diagrams for neural networks,and the use of evolutionary computing tools for analysis.An example of using particle swarm to develop an explanation facility is included in this chapter. The book concludes with Chapter 12,Case Study Summaries,which provides examples of practical applications.This "virtual"chapter is located on the book's web site.Having it there makes it a"living"chapter that can be updated periodi- cally.We will add new case studies from time to time and delete older ones as they become obsolete.We invite you,the reader,to submit case studies you would like to have considered for inclusion.(Please see the web site for more information about this.)Among the initial case studies posted are two based on recent work by us, the authors,including one on human EEG analysis and another on optimization of logistics operations.Other case studies discussed in detail are schedule optimization and control system design.Several other case study examples are briefly reviewed. A bibliography concludes the book.The glossary is a"virtual"one that is located, with Chapter 12,on this book's web site http://www.computelligence.org/issue/CICI/ CICI.html. Our Approach:What This Book Is,and Is Not,About This book asserts that computational intelligence rests on a foundation ofevolution- ary computation.This is certainly not the only way to view computational intelli- gence,but so far in the authors'experience,it has proved useful and effective. It is about computational tools that you can use in practical applications.Although the authors have backgrounds in engineering and computer science,CI tools are just as applicable to problems in other fields such as cognitive science and business

Preface ~ ~) concludes with a look at some design considerations and special topics related to fuzzy systems. Chapter 8, Fuzzy System Implementations, discusses factors to consider when implementing fuzzy systems and presents two implementation examples: a tradi￾tional fuzzy rule system and an evolutionary fuzzy rule system. The evolutionary fuzzy rule system provides a transition into computational intelligence systems. Chapter 9, Computational Intelligence Implementations, reflects recent devel￾opments in the field, including evolutionary fuzzy systems and approaches to sys￾tem adaptation using computational intelligence. We expand the discussion of the interaction and cooperation among the three basic components of CI and include a section on adaptive evolutionary computation using fuzzy systems. Chapter 10, Performance Metrics, includes a number of system performance measures not generally used in other disciplines. Included are percent correct, sum￾squared error, absolute error, normalized error, receiver operating characteristic curves, recall and precision, confusion matrices, and the chi-squared test. Chapter 11, Analysis and Explanation, presents several tools that are helpful in assessing and explaining how well a computational intelligence tool is doing its job. Included are sensitivity analyses, Hinton diagrams for neural networks, and the use of evolutionary computing tools for analysis. An example of using particle swarm to develop an explanation facility is included in this chapter. The book concludes with Chapter 12, Case Study Summaries, which provides examples of practical applications. This "virtual" chapter is located on the book's web site. Having it there makes it a "living" chapter that can be updated periodi￾cally. We will add new case studies from time to time and delete older ones as they become obsolete. We invite you, the reader, to submit case studies you would like to have considered for inclusion. (Please see the web site for more information about this.) Among the initial case studies posted are two based on recent work by us, the authors, including one on human EEG analysis and another on optimization of logistics operations. Other case studies discussed in detail are schedule optimization and control system design. Several other case study examples are briefly reviewed. A bibliography concludes the book. The glossary is a "virtual" one that is located, with Chapter 12, on this book's web site http://www.computelligence.org/issue/CICI/ CICI.html. Our Approach" What This Book is, and Is Not, About This book asserts that computational intelligence rests on a foundation of evolution￾ary computation. This is certainly not the only way to view computational intelli￾gence, but so far in the authors' experience, it has proved useful and effective. It is about computational tools that you can use in practical applications. Although the authors have backgrounds in engineering and computer science, CI tools are just as applicable to problems in other fields such as cognitive science and business

Preface This book is about self-organization,which is closely related to emergent computation.Self-organization involves simple processes that lead to complex results,and the whole being greater than the sum of its parts.As Stephen Wolfram (1994)said,"It is possible to make things of great complexity out of things that are very simple.There is no conservation of simplicity." It is about complex adaptive systems,a term that describes nonlinear systems com- prising the interaction of numerous adaptive elements,or entities.The concepts of self-organization and complexity are related,as we discuss later. This book is not an exhaustive treatise on all permutations and variations ofcom- putational intelligence and its constituent methodologies.If you want an exhaustive discussion of artificial neural network paradigms,for instance,you'll need to turn to another book.We present only those paradigms we believe provide the most useful tools for someone solving practical problems. It is not a compendium of mathematical derivations and proofs.We present only those few we believe are essential to gaining a working-level understanding of how and why the computational tools work. This book is not about agents.Most of our computational intelligence tools do not qualify as"agents"because they lack the required autonomy and specialization. They can,however,be incorporated into intelligent agents and agent systems. It is not about life.We nip around the edges of artificial life in a few places,but we don't address the question"What is alive?"(We do,however,share some pre- liminary thoughts on that subject.)We also do not address the search for artificial intelligence (whatever that is)or even for a computational intelligence tool from which intelligent behavior will emerge.Our focus is on solving problems. Throughout the text,additional aspects of our approach and philosophy should become evident,perhaps a little bit at a time.First,when considering computa- tional intelligence tools and systems,traditional distinctions between hardware and software get a bit blurred;distinctions between data and program are often almost nonexistent.Second,our emphasis is on problem solving and applications rather than physiological,biological,or behavioral plausibility.We do not pay too much attention to whether the CI tools reflect what actually goes on in the brain or any other part of a biological organism.Third,we believe that the activities of a com- putational intelligence application developer and user are often somewhat different from those in other technical areas. Developing computational intelligence applications requires the developer to play two roles.The first is the hands-on active design,develop,test,and debug role that is fairly common in other technical areas.The second,as important as the first, is a more passive observation and analytical thinking role.Results from a compu- tational intelligence tool are often not what was expected.Most of the time,if the developer takes the time to observe and think,rather than "bash to fit and paint to match,"something very useful can be learned

Preface This book is about self-organization, which is closely related to emergent computation. Self-organization involves simple processes that lead to complex results, and the whole being greater than the sum of its parts. As Stephen Wolfram (1994) said, "It is possible to make things of great complexity out of things that are very simple. There is no conservation of simplicity." It is about complex adaptive systems, a term that describes nonlinear systems com￾prising the interaction of numerous adaptive elements, or entities. The concepts of self-organization and complexity are related, as we discuss later. This book is not an exhaustive treatise on all permutations and variations of com￾putational intelligence and its constituent methodologies. If you want an exhaustive discussion of artificial neural network paradigms, for instance, you'll need to turn to another book. We present only those paradigms we believe provide the most useful tools for someone solving practical problems. It is not a compendium of mathematical derivations and proofs. We present only those few we believe are essential to gaining a working-level understanding of how and why the computational tools work. This book is not about agents. Most of our computational intelligence tools do not qualify as "agents" because they lack the required autonomy and specialization. They can, however, be incorporated into intelligent agents and agent systems. It is not about life. We nip around the edges of artificial life in a few places, but we don't address the question "What is alive?" (We do, however, share some pre￾liminary thoughts on that subject.) We also do not address the search for artificial intelligence (whatever that is) or even for a computational intelligence tool from which intelligent behavior will emerge. Our focus is on solving problems. Throughout the text, additional aspects of our approach and philosophy should become evident, perhaps a little bit at a time. First, when considering computa￾tional intelligence tools and systems, traditional distinctions between hardware and software get a bit blurred; distinctions between data and program are often almost nonexistent. Second, our emphasis is on problem solving and applications rather than physiological, biological, or behavioral plausibility. We do not pay too much attention to whether the CI tools reflect what actually goes on in the brain or any other part of a biological organism. Third, we believe that the activities of a com￾putational intelligence application developer and user are often somewhat different from those in other technical areas. Developing computational intelligence applications requires the developer to play two roles. The first is the hands-on active design, develop, test, and debug role that is fairly common in other technical areas. The second, as important as the first, is a more passive observation and analytical thinking role. Results from a compu￾tational intelligence tool are often not what was expected. Most of the time, if the developer takes the time to observe and think, rather than "bash to fit and paint to match," something very useful can be learned

Preface Web Site Details The authors'web site for this book is http://www.computelligence.org/issue/CICI/ CICI.html.(There is a link to this site from the publisher's web site.)Software imple- mentations are written for the Windows and/or Java environment,and executable versions of software described in the implementation chapters are located and main- tained on the web site.Included as part of each implementation are the ancillary files-a run file and a data file-needed to run the implementation.In addition,out- put(results)files,obtained by the authors using the executable and ancillary files, are provided.You may want to rename these output files,or move them to another directory,so that you can compare your results with those of the authors. We'd like to emphasize that the software is not just for demonstration;you can use it for many real-world applications.The Cand C++source code has been written using the Borland C++4.5 development environment.The Java code will run on any computer that supports the Java Virtual Machine;this includes machines running Windows,Unix,and Macintosh operating systems. Of special note are the recent variations of particle swarm optimization that have been integrated into the EC theory and paradigms chapter and the EC implementa- tions chapter.Source code is provided on the web site for some of the implementa- tions so that you can modify the software for specific applications. Some of our software can be run using a web browser.Other software,including source code,is useful only after downloading it from the book's web site.Approx- imately 600 slides that cover the material in this book are available to instructors (or anyone else)at no cost.These slides,configured as Word files,are downloadable from the web site.The site also contains hyperlinks to other resource information on the Internet related to subjects in this book. A significant amount of source code is also on the web site.A total of eight software modules are available,both as executables and as source code: ■Genetic algorithm Particle swarm optimization(including multiple swarms) ■ Back-propagation neural network Learning vector quantization neural network Self-organizing feature map neural network Evolutionary back-propagation neural network ■Fuzzy rule system Evolutionary fuzzy rule system We ask that you send the authors a payment of US $25 per software module of source code($150 for all of the source code)if you find it useful.We are relying on your honesty.(The address is on the web site with the software.)

Preface ~.,~~ Web Site Details The authors' web site for this book is http://www.computelligence.org/issue/CICI/ CICI.html. (There is a link to this site from the publisher's web site.) Software imple￾mentations are written for the Windows and/or lava environment, and executable versions of software described in the implementation chapters are located and main￾tained on the web site. Included as part of each implementation are the ancillary files~a run file and a data file~needed to run the implementation. In addition, out￾put (results) files, obtained by the authors using the executable and ancillary files, are provided. You may want to rename these output files, or move them to another directory, so that you can compare your results with those of the authors. We'd like to emphasize that the software is not just for demonstration; you can use it for many real-world applications. The C and C++ source code has been written using the Borland C++ 4.5 development environment. The lava code will run on any computer that supports the lava Virtual Machine; this includes machines running Windows, Unix, and Macintosh operating systems. Of special note are the recent variations of particle swarm optimization that have been integrated into the EC theory and paradigms chapter and the EC implementa￾tions chapter. Source code is provided on the web site for some of the implementa￾tions so that you can modify the software for specific applications. Some of our software can be run using a web browser. Other software, including source code, is useful only after downloading it from the book's web site. Approx￾imately 600 slides that cover the material in this book are available to instructors (or anyone else) at no cost. These slides, configured as Word files, are downloadable from the web site. The site also contains hyperlinks to other resource information on the Internet related to subjects in this book. A significant amount of source code is also on the web site. A total of eight software modules are available, both as executables and as source code: m Genetic algorithm m Particle swarm optimization (including multiple swarms) m Back-propagation neural network m Learning vector quantization neural network [] Self-organizing feature map neural network [] Evolutionary back-propagation neural network [] Fuzzy rule system . Evolutionary fuzzy rule system We ask that you send the authors a payment of US $25 per software module of source code ($150 for all of the source code) if you find it useful. We are relying on your honesty. (The address is on the web site with the software.)

Preface Finally,as described previously,Chapter 12,Case Studies,is available on the web site. Acknowledgments Each of us has numerous people who should be acknowledged;we mention only a few. Russ Eberhart:First,I want to acknowledge my wife Francie and son Sean who put up with a higher than usual absence rate of their spouse and father,respectively. I also want to acknowledge my son Mark,a three-time cancer survivor,who has taught me what courage is.Special thanks go to my students in ECE 536,Intro- duction to Computational Intelligence.They were the guinea pigs.Sometimes,just from their eyes glazing over,I knew that a section needed to be rewritten(or deleted). Their patience is appreciated,and their input has been invaluable. Yuhui Shi:I would like to thank my parents and parents-in-law for taking good care of my daughter Melissa Xueyin Shi and my son Nicholas Yuge Shi so that I had plenty of quality time to work on this book.My thanks also go to profes- sors Zhenya He of Southeast University,M.N.S.Swamy and M.Omair Ahmad of Concordia University,Xin Yao ofthe University of Birmingham,Jinhyung Kim ofthe Korean Advanced Institute of Science and Technology,and to Russell C.Eberhart, who are my mentors and have paved the way for me in my career development. Both of us acknowledge the contributions of our technical reviewers.Their insights resulted in improvements in both the organization and content of this book. Finally,we are grateful to the team at Morgan Kaufmann Publishers who worked dili- gently with us throughout the process of writing,editing,and production.Working with Denise Penrose,Diane Cerra,Emilia Thiuri,Marilyn Rash,and Mary James has been a pleasure and a learning experience

Preface Finally, as described previously, Chapter 12, Case Studies, is available on the web site. Acknowledgments Each of us has numerous people who should be acknowledged; we mention only a few. Russ Eberhart:. First, I want to acknowledge my wife Francie and son Sean who put up with a higher than usual absence rate of their spouse and father, respectively. I also want to acknowledge my son Mark, a three-time cancer survivor, who has taught me what courage is. Special thanks go to my students in ECE 536, Intro￾duction to Computational Intelligence. They were the guinea pigs. Sometimes, just from their eyes glazing over, I knew that a section needed to be rewritten (or deleted). Their patience is appreciated, and their input has been invaluable. Yuhui Shi: I would like to thank my parents and parents-in-law for taking good care of my daughter Melissa Xueyin Shi and my son Nicholas Yuge Shi so that I had plenty of quality time to work on this book. My thanks also go to profes￾sors Zhenya He of Southeast University, M. N. S. Swamy and M. Omair Ahmad of Concordia University, Xin Yao of the University of Birmingham, Jinhyung Kim ofthe Korean Advanced Institute of Science and Technology, and to Russell C. Eberhart, who are my mentors and have paved the way for me in my career development. Both of us acknowledge the contributions of our technical reviewers. Their insights resulted in improvements in both the organization and content of this book. Finally, we are grateful to the team at Morgan Kaufmann Publishers who worked dili￾gently with us throughout the process of writing, editing, and production. Working with Denise Penrose, Diane Cerra, Emilia Thiuri, Marilyn Rash, and Mary James has been a pleasure and a learning experience

chapter one Foundations This chapter introduces general terms the differences between biological struc- used to discuss computational intelligence tures and these analysis tools,and we as well as component methodologies- discuss the behavioral motivations for fuzzy computational intelligence (CI),including systems.The chapter ends with a review of artificial neural networks,fuzzy logic,and myths related to implementations and appli- evolutionarycomputation-asthey are used cations of CI and its component technolo- in this text.We review the biological bases gies,and a review of major application areas for artificial neural network and evolution- for each of the three main computational ary computation analysis tools,including intelligence methodologies. ◆ 1

chapter one Foundations This chapter introduces general terms used to discuss computational intelligence as well as component methodologies~ computational intelligence (Cl), including artificial neural networks, fuzzy logic, and evolutionarycomputation~as they are used in this text. We review the biological bases for artificial neural network and evolution￾ary computation analysis tools, including the differences between biological struc￾tures and these analysis tools, and we discuss the behavioral motivations for fuzzy systems. The chapter ends with a review of myths related to implementations and appli￾cations of Cl and its component technolo￾gies, and a review of major application areas for each of the three main computational intelligence methodologies, m

Chapter One-Foundations Definitions This section defines some of the most important terms used in this book.These definitions set the stage for more detailed analyses;more comprehensive definitions appear in subsequent chapters.Often,the first time a term is used in the book,it is in italics.In addition,whenever a term is italicized,you can find its definition in the glossary. We begin with a general definition of intelligence and then focus on the issues relevant to computational intelligence.A standard dictionary(Webster's New Colle- giate Dictionary,1975)definition of intelligence is:"1 a(1):The ability to learn or understand or to deal with new or trying situations:REASON;also:the skilled use of reason(2):the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria(as tests)." "Intelligence is the capability ofa system to adapt its behavior to meet its goals in a range ofenvironments.It is a property of all purpose-driven decision-makers."This definition,perhaps more relevant to the subject matter of this book,was published by David Fogel (1995). An artificial neural network(ANN)is an analysis paradigm that is roughly mod- eled after the massively parallel structure of the brain.It simulates a highly inter- connected,parallel computational structure with many relatively simple individual processing elements(PEs).Henceforth in this text the terms artificial neural network and neural network are used interchangeably. As used in this text,fuzziness refers to nonstatistical imprecision and vagueness in information and data.Most concepts dealt with or described in the"real world" are fuzzy.For example,"It is kind of foggy outside now,but it should be fairly sunny before too long"is an example of a statement that incorporates three fuzzy concepts: “kind of,”“fairly,”and“before too long..”(It could even be argued that the word now"is imprecise and vague enough to be fuzzy.) Fuzzy sets model the properties of imprecision,approximation,or vagueness.In conventionallogic,known as crisp logic,an element either is or is not a member ofthe set.It can be said,therefore,that each element has a membership value of either 1 or 0 in the set.In a fuzzy set,fuzzy membership values reflect the membership extents (or grades)of the elements in the set.It will be shown that a membership function is the basic idea in fuzzy set theory;a fuzzy membership function is identical to a fuzzy set. Fuzzy logic is the logic of"approximate reasoning."It comprises operations on fuzzy sets including equality,containment,complementation,intersection,and union;it is a generalization of conventional (two-valued,or crisp)logic. Evolutionary computation comprises machine learning optimization and clas- sification paradigms roughly based on mechanisms of evolution such as biolog- ical genetics and natural selection.The evolutionary computation field includes genetic algorithms,evolutionary programming,genetic programming,evolution

Chapter One--Foundations Definitions This section defines some of the most important terms used in this book. These definitions set the stage for more detailed analyses; more comprehensive definitions appear in subsequent chapters. Often, the first time a term is used in the book, it is in italics. In addition, whenever a term is italicized, you can find its definition in the glossary. We begin with a general definition of intelligence and then focus on the issues relevant to computational intelligence. A standard dictionary (Webster's New Colle￾giate Dictionary, 1975) definition of intelligence is: "1 a (1): The ability to learn or understand or to deal with new or trying situations : REASON; also : the skilled use of reason (2): the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (as tests)." "Intelligence is the capability of a system to adapt its behavior to meet its goals in a range of environments. It is a property of all purpose-driven decision-makers." This definition, perhaps more relevant to the subject matter of this book, was published by David Fogel (1995). An artificial neural network (ANN) is an analysis paradigm that is roughly mod￾eled after the massively parallel structure of the brain. It simulates a highly inter￾connected, parallel computational structure with many relatively simple individual processing elements (PEs). Henceforth in this text the terms artificial neural network and neural network are used interchangeably. As used in this text, fuzziness refers to nonstatistical imprecision and vagueness in information and data. Most concepts dealt with or described in the "real world" are fuzzy. For example, "It is kind of foggy outside now, but it should be fairly sunny before too long" is an example of a statement that incorporates three fuzzy concepts: "kind of," "fairly," and "before too long." (It could even be argued that the word "now" is imprecise and vague enough to be fuzzy.) Fuzzy sets model the properties of imprecision, approximation, or vagueness. In conventional logic, known as crisp logic, an element either is or is not a member ofthe set. It can be said, therefore, that each element has a membership value of either 1 or 0 in the set. In a fuzzy set, fuzzy membership values reflect the membership extents (or grades) of the elements in the set. It will be shown that a membership function is the basic idea in fuzzy set theory; a fuzzy membership function is identical to a fuzzy set. Fuzzy logic is the logic of "approximate reasoning." It comprises operations on fuzzy sets including equality, containment, complementation, intersection, and union; it is a generalization of conventional (two-valued, or crisp) logic. Evolutionary computation comprises machine learning optimization and clas￾sification paradigms roughly based on mechanisms of evolution such as biolog￾ical genetics and natural selection. The evolutionary computation field includes genetic algorithms, evolutionary programming, genetic programming, evolution