Support Vector Machines in R the fraction of support vectors found in the data set,thus controlling the complexity of the classification function build by the SVM(see Appendix for details). For multi-class classification,mostly voting schemes such as one-against-one and one-against- all are used.In the one-against-all method k binary SVM classifiers are trained,where k is the number of classes,each trained to separate one class from the rest.The classifiers are then combined by comparing their decision values on a test data instance and labeling it according to the classifier with the highest decision value. In the one-against-one classification method (also called pairwise classification;see Knerr, Personnaz,and Dreyfus 1990;Krefel 1999),()classifiers are constructed where each one is trained on data from two classes.Prediction is done by voting where each classifier gives a prediction and the class which is most frequently predicted wins ("Max Wins").This method has been shown to produce robust results when used with SVMs(Hsu and Lin 2002a). Although this suggests a higher number of support vector machines to train the overall CPU time used is less compared to the one-against-all method since the problems are smaller and the SVM optimization problem scales super-linearly. Furthermore,SVMs can also produce class probabilities as output instead of class labels.This is can done by an improved implementation (Lin,Lin,and Weng 2001)of Platt's a posteriori probabilities (Platt 2000)where a sigmoid function 1 P(y=1|f)=1+eA+B (8) is fitted to the decision values f of the binary SVM classifiers,A and B being estimated by minimizing the negative log-likelihood function.This is equivalent to fitting a logistic regression model to the estimated decision values.To extend the class probabilities to the multi-class case,all binary classifiers class probability output can be combined as proposed in Wu,Lin,and Weng (2003). In addition to these heuristics for extending a binary SVM to the multi-class problem,there have been reformulations of the support vector quadratic problem that deal with more than two classes.One of the many approaches for native support vector multi-class classification is the one proposed in Crammer and Singer(2000),which we will refer to as 'spoc-svc'.This algorithm works by solving a single optimization problem including the data from all classes. The primal formulation is: minimize a0=∑wP+2s n=1 m subject to (Φ(x),w）-(④(x),wn〉≥b-a (i=1....,m) (9) where b}=1-δ，n (10) where the decision function is argmaxn=1,…k(Φ(x),wn) (11) Details on performance and benchmarks on various approaches for multi-class classification can be found in Hsu and Lin (2002b).4 Support Vector Machines in R the fraction of support vectors found in the data set, thus controlling the complexity of the classification function build by the SVM (see Appendix for details). For multi-class classification, mostly voting schemes such as one-against-one and one-against￾all are used. In the one-against-all method k binary SVM classifiers are trained, where k is the number of classes, each trained to separate one class from the rest. The classifiers are then combined by comparing their decision values on a test data instance and labeling it according to the classifier with the highest decision value. In the one-against-one classification method (also called pairwise classification; see Knerr, Personnaz, and Dreyfus 1990; Kreßel 1999), ￾ k 2
classifiers are constructed where each one is trained on data from two classes. Prediction is done by voting where each classifier gives a prediction and the class which is most frequently predicted wins (“Max Wins”). This method has been shown to produce robust results when used with SVMs (Hsu and Lin 2002a). Although this suggests a higher number of support vector machines to train the overall CPU time used is less compared to the one-against-all method since the problems are smaller and the SVM optimization problem scales super-linearly. Furthermore, SVMs can also produce class probabilities as output instead of class labels. This is can done by an improved implementation (Lin, Lin, and Weng 2001) of Platt’s a posteriori probabilities (Platt 2000) where a sigmoid function P(y = 1 | f) = 1 1 + eAf+B (8) is fitted to the decision values f of the binary SVM classifiers, A and B being estimated by minimizing the negative log-likelihood function. This is equivalent to fitting a logistic regression model to the estimated decision values. To extend the class probabilities to the multi-class case, all binary classifiers class probability output can be combined as proposed in Wu, Lin, and Weng (2003). In addition to these heuristics for extending a binary SVM to the multi-class problem, there have been reformulations of the support vector quadratic problem that deal with more than two classes. One of the many approaches for native support vector multi-class classification is the one proposed in Crammer and Singer (2000), which we will refer to as ‘spoc-svc’. This algorithm works by solving a single optimization problem including the data from all classes. The primal formulation is: minimize t({wn}, ξ) = 1 2 X k n=1 kwnk 2 + C m Xm i=1 ξi subject to hΦ(xi), wyi i − hΦ(xi), wni ≥ b n i − ξi (i = 1, . . . , m) (9) where b n i = 1 − δyi,n (10) where the decision function is argmaxn=1,...,khΦ(xi), wni (11) Details on performance and benchmarks on various approaches for multi-class classification can be found in Hsu and Lin (2002b)