2.6.3 Equivalence of Ll and L2 Support Vector Machines.. 67 2.6.4 Nonunique Solutions 5.5 Reducing the Number of Support Vectors 2.6.6 Degenerate Solutions 2.6.7 Duplicate Copies of Data 2. 6.8 Imbalanced Data 2.6.9 Classification for the blood Cell Data 2. 7 Class Boundaries for Different Kernels 2.8 Developing Classifiers 2.8.1 Model Selection 2.8.2 Estimating Generalization Errors 2.8.3 Sophistication of Model Selection 89337 2.8.4 Effect of Model Selection by Cross-Validation 2. 9 Invariance for Linear transformation References Iticlass Support Vector Machines 113 One-Against-All Support Vector Machines 114 3.1.1 Conventional Support Vector Machines 114 3.1.2 Fuzzy Support Vector Machines 3.1.3 Equivalence of Fuzzy Support Vector Machines and Support Vector Machines with Continuous Decision 119 3.1.4 Decision-Tree-Based Support Vector Machines 3.2 Pairwise Support Vector Machines 3.2.1 Conventional Support Vector Machines 3.2.2 Fuzzy Support Vector Machines 128 3.2.3 Performance Comparison of Fuzzy Support Vector Machines 3.2.4 Cluster-Based Support Vector Machines 3.2.5 Decision-Tree-Based Support Vector Machines 133 3.2.6 Pairwise Classification with Correcting Classifiers 3.3 Error-Correcting Output Codes 144 3.3.1 Output Coding by Error-Correcting Codes 3.3.2 Unified Scheme for Output Coding 3.3.3 Equivalence of ECOC with Membership Functions.. 147 3.3.4 Performance Evaluation 147 3.4 All-at-Once Support Vector Machines 149 3.5 Comparisons of Architectures 152 3.5.1 One-Against-All Support Vector Machines 152 3.5.2 Pairwise Support Vector Machines 152 3.5.3 ECOC Support Vector Machines 3.5.4 All-at-Once Support Vector Machines 153 5.5 Training Difficulty 3.5.6 Training Time Comp arIson References 158xiv Contents 2.6.3 Equivalence of L1 and L2 Support Vector Machines . . . 67 2.6.4 Nonunique Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 2.6.5 Reducing the Number of Support Vectors . . . . . . . . . . . 78 2.6.6 Degenerate Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 2.6.7 Duplicate Copies of Data . . . . . . . . . . . . . . . . . . . . . . . . . . 83 2.6.8 Imbalanced Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 2.6.9 Classification for the Blood Cell Data . . . . . . . . . . . . . . . 85 2.7 Class Boundaries for Different Kernels . . . . . . . . . . . . . . . . . . . . 88 2.8 Developing Classifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 2.8.1 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 2.8.2 Estimating Generalization Errors . . . . . . . . . . . . . . . . . . . 93 2.8.3 Sophistication of Model Selection . . . . . . . . . . . . . . . . . . . 97 2.8.4 Effect of Model Selection by Cross-Validation . . . . . . . . 98 2.9 Invariance for Linear Transformation . . . . . . . . . . . . . . . . . . . . . . 102 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 3 Multiclass Support Vector Machines . . . . . . . . . . . . . . . . . . . . . . 113 3.1 One-Against-All Support Vector Machines . . . . . . . . . . . . . . . . . 114 3.1.1 Conventional Support Vector Machines . . . . . . . . . . . . . . 114 3.1.2 Fuzzy Support Vector Machines . . . . . . . . . . . . . . . . . . . . 116 3.1.3 Equivalence of Fuzzy Support Vector Machines and Support Vector Machines with Continuous Decision Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 3.1.4 Decision-Tree-Based Support Vector Machines . . . . . . . 122 3.2 Pairwise Support Vector Machines . . . . . . . . . . . . . . . . . . . . . . . . 127 3.2.1 Conventional Support Vector Machines . . . . . . . . . . . . . . 127 3.2.2 Fuzzy Support Vector Machines . . . . . . . . . . . . . . . . . . . . 128 3.2.3 Performance Comparison of Fuzzy Support Vector Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 3.2.4 Cluster-Based Support Vector Machines . . . . . . . . . . . . . 132 3.2.5 Decision-Tree-Based Support Vector Machines . . . . . . . 133 3.2.6 Pairwise Classification with Correcting Classifiers. . . . . 143 3.3 Error-Correcting Output Codes . . . . . . . . . . . . . . . . . . . . . . . . . . 144 3.3.1 Output Coding by Error-Correcting Codes. . . . . . . . . . . 145 3.3.2 Unified Scheme for Output Coding . . . . . . . . . . . . . . . . . 146 3.3.3 Equivalence of ECOC with Membership Functions . . . . 147 3.3.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 3.4 All-at-Once Support Vector Machines . . . . . . . . . . . . . . . . . . . . . 149 3.5 Comparisons of Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 3.5.1 One-Against-All Support Vector Machines . . . . . . . . . . . 152 3.5.2 Pairwise Support Vector Machines . . . . . . . . . . . . . . . . . . 152 3.5.3 ECOC Support Vector Machines . . . . . . . . . . . . . . . . . . . 153 3.5.4 All-at-Once Support Vector Machines . . . . . . . . . . . . . . . 153 3.5.5 Training Difficulty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 3.5.6 Training Time Comparison . . . . . . . . . . . . . . . . . . . . . . . . 157 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158