Preface Preface to the first Edition I was shocked to see a student's report on performance comparisons between support vector machines(SVMs)and fuzzy classifiers that we had developed with our best endeavors. Classification performance of our fuzzy classifiers was comparable, but in most cases inferior, to that of support vector ma- chines. This tendency was especially evident when the numbers of class data were small. I shifted my research efforts from developing fuzzy classifiers with gh generalization ability to developing support vector machine-based clas- sifiers This book focuses on the application of support vector machines to ern classification. Specifically, we discuss the properties of support vector machines that are useful for pattern classification applications, several multi- class models, and variants of support vector machines. To clarify their appli- cability to real-world problems, we compare the performance of most models discussed in the book using real-world benchmark data. Readers interested in the theoretical aspect of support vector machines should refer to book ch as 1-4 Three-layer neural networks are universal classifiers in that they can clas- sify any labeled data correctly if there are no identical data in different classes 5, 6. In training multi layer neural network classifiers, network weights are usually corrected so that the sum-of-squares error between the network out puts and the desired outputs is minimized. But because the decision bound aries between classes acquired by training are not directly determined, clas- sification performance for the unknown data, i.e., the generalization abilit depends on the training method. And it degrades greatly when the number of training data is small and there class On the other hand, in training support vector machines the decision boundaries are determined directly from the training data so that the sepa- rating margins of decision boundaries are maximized in the high-dimensional space called feature space. This learning strategy, based on statistical learning theory developed by Vapnik [ 1, 2 minimizes the classification errors of the training data and the unknown data sion boundaries is introduced to non-SVM-type classifiers, their pefa oo,or Therefore, the generalization abilities of support vector machines and other classifiers differ significantly, especially when the number of training data is small. This means that if some mechanism to maximize the margins degradation will be prevented when the class overlap is scarce or nonexistent. I In the original support vector machine, an n-class classification problem is converted into n two-class problems, and in the ith two-class problem we determine the optimal decision function that separates class i from the remaining classes. In classification, if one of the n decision functions classifies complexity of the classifier, is added to the objective functionPreface vii Preface to the First Edition I was shocked to see a student’s report on performance comparisons between support vector machines (SVMs) and fuzzy classifiers that we had developed with our best endeavors. Classification performance of our fuzzy classifiers was comparable, but in most cases inferior, to that of support vector ma￾chines. This tendency was especially evident when the numbers of class data were small. I shifted my research efforts from developing fuzzy classifiers with high generalization ability to developing support vector machine-based clas￾sifiers. This book focuses on the application of support vector machines to pat￾tern classification. Specifically, we discuss the properties of support vector machines that are useful for pattern classification applications, several multi￾class models, and variants of support vector machines. To clarify their appli￾cability to real-world problems, we compare the performance of most models discussed in the book using real-world benchmark data. Readers interested in the theoretical aspect of support vector machines should refer to books such as [1–4]. Three-layer neural networks are universal classifiers in that they can clas￾sify any labeled data correctly if there are no identical data in different classes [5, 6]. In training multilayer neural network classifiers, network weights are usually corrected so that the sum-of-squares error between the network out￾puts and the desired outputs is minimized. But because the decision bound￾aries between classes acquired by training are not directly determined, clas￾sification performance for the unknown data, i.e., the generalization ability, depends on the training method. And it degrades greatly when the number of training data is small and there is no class overlap. On the other hand, in training support vector machines the decision boundaries are determined directly from the training data so that the sepa￾rating margins of decision boundaries are maximized in the high-dimensional space called feature space. This learning strategy, based on statistical learning theory developed by Vapnik [1, 2], minimizes the classification errors of the training data and the unknown data. Therefore, the generalization abilities of support vector machines and other classifiers differ significantly, especially when the number of training data is small. This means that if some mechanism to maximize the margins of deci￾sion boundaries is introduced to non-SVM-type classifiers, their performance degradation will be prevented when the class overlap is scarce or nonexistent.1 In the original support vector machine, an n-class classification problem is converted into n two-class problems, and in the ith two-class problem we determine the optimal decision function that separates class i from the remaining classes. In classification, if one of the n decision functions classifies 1 To improve generalization ability of a classifier, a regularization term, which controls the complexity of the classifier, is added to the objective function