heme Feature Artificial Neural Networks:A Tutorial Anil K.Jain umerous advances have been made in developing intelligent Michigan State University systems,some inspired by biological neural networks. Researchers from many scientific disciplines are designing arti- Jianchang Mao ficial neural networks (ANNs)to solve a variety of problems in pattern K.M.Mohiuddin recognition,prediction,optimization,associative memory,and control IBM Almaden Research Center (see the“Challenging problems'”sidebar). Conventional approaches have been proposed for solving these prob- lems.Although successful applications can be found in certain well-con- strained environments,none is flexible enough to perform well outside its domain.ANNs provide exciting alternatives,and many applications could benefit from using them.3 This article is for those readers with little or no knowledge of ANNs to help them understand the other articles in this issue of Computer.We dis- cuss the motivations behind the development of ANNs,describe the basic biological neuron and the artificial computational model,outline net- work architectures and learning processes,and present some of the most commonly used ANN models.We conclude with character recognition.a successful ANN application. WHY ARTIFICIAL NEURAL NETWORKS? The long course of evolution has given the human brain many desir able characteristics not present in von Neumann or modern parallelcom- puters.These include massive parallelism, distributed representation and computation, ·learning ability, generalization ability, 。adaptivity, .inherent contextual information processing, fault tolerance,and low energy consumption. 烧 It is hoped that devices based on biological neural networks will possess some of these desirable characteristics. These massively parallel Modern digital computers outperform humans in the domain of numeric computation and related symbol manipulation.However, systems with large numbers humans can effortlessly solve complex perceptual problems (like recog- nizing a man in a crowd from a mere glimpse of his face)at such a high of interconnected simple speed and extent as to dwarf the world's fastest computer.Why is there such a remarkable difference in their performance?The biological neural processors may solve a variety system architecture is completely different from the von Neumann archi- tecture(see Table 1).This difference significantly affects the type of func- of challenging computational tions each computational model can best perform. Numerous efforts to develop "intelligent"programs based on von problems.This tutorial Neumann's centralized architecture have not resulted in general-purpose intelligent programs.Inspired by biological neural networks,ANNs are provides the background and massively parallel computing systems consisting of an exremely large num- ber of simple processors with many interconnections.ANN models attempt the basics. to use some"organizational"principles believed to be used in the human 0018-91629655.00.1996IEEE March 1996 B0Ani1 K. Jain Michigan State University Jianchang Mao K.M. Mohiuddin ZBMAZmaden Research Center umerous advances have been made in developing intelligent N systems, some inspired by biological neural networks. Researchers from many scientific disciplines are designing arti￾ficial neural networks (A”s) to solve a variety of problems in pattern recognition, prediction, optimization, associative memory, and control (see the “Challenging problems” sidebar). Conventional approaches have been proposed for solving these prob￾lems. Although successful applications can be found in certain well-con￾strained environments, none is flexible enough to perform well outside its domain. ANNs provide exciting alternatives, and many applications could benefit from using them.’ This article is for those readers with little or no knowledge of ANNs to help them understand the other articles in this issue of Computer. We dis￾cuss the motivations behind the development of A”s, describe the basic biological neuron and the artificial computational model, outline net￾work architectures and learning processes, and present some of the most commonly used ANN models. We conclude with character recognition, a successful ANN application. WHY ARTIFICIAL NEURAL NETWORKS? The long course of evolution has given the human brain many desir￾able characteristics not present invon Neumann or modern parallel com￾puters. These include These massively parallel systems with large numbers of interconnected simple processors may solve a variety of challenging computational problems. This tutorial provides the background and the basics. massive parallelism, distributed representation and computation, learning ability, generalization ability, adaptivity, inherent contextual information processing, fault tolerance, and low energy consumption. It is hoped that devices based on biological neural networks will possess some of these desirable characteristics. Modern digital computers outperform humans in the domain of numeric computation and related symbol manipulation. However, humans can effortlessly solve complex perceptual problems (like recog￾nizing a man in a crowd from a mere glimpse of his face) at such a high speed and extent as to dwarf the world’s fastest computer. Why is there such a remarkable difference in their performance? The biological neural system architecture is completely different from the von Neumann archi￾tecture (see Table l). This difference significantly affects the type of func￾tions each computational model can best perform. Numerous efforts to develop “intelligent” programs based on von Neumann’s centralized architecture have not resulted in general-purpose intelligent programs. Inspired by biological neural networks, ANNs are massively parallel computing systems consisting of an exremely large num￾ber of simple processors with many interconnections. ANN models attempt to use some “organizational” principles believed to be used in the human 0018-9162/96/$5.000 1996 IEEE March 1996