SVM Tutorial 3 boundaries demarcating the classes (Why?We want to be as sure as possible that we are not making classification mistakes,and thus we want our data points from the two classes to lie as far away from each other as possible).This distance is called the margin,so what we want to do is to obtain the marimal margin. Thus,we want to maximizewhichsequivalent to minimizing 2 which is in turn equivalent to minimizing(since square root is a monotonic 2 function). This quadratic programming problem is expressed as: minw.空 subject to:yi(wxi+b)>1 (V data points xi). 2 Soft-margin extention Consider the case that your data isn't perfectly linearly separable.For instance, maybe you aren't guaranteed that all your data points are correctly labelled,so you want to allow some data points of one class to appear on the other side of the boundary. We can introduce slack variables-an ei>0 for each xi.Our quadratic program- ming problem becomes: minw.,e+C∑：G subject to:y(wTx+b)≥1-ei and ei≥0（付data points xi. 3 Nonlinear decision boundary Mapping your data vectors,xi,into a higher-dimension(even infinite)feature space may make them linearly separable in that space (whereas they may not be linearly separable in the original space).The formulation of the quadratic programming problem is as above,but with all xi replaced with o(xi),where o provides the higher-dimensional mapping.So we have the standard SVM formu- lation:SVM Tutorial 3 boundaries demarcating the classes (Why? We want to be as sure as possible that we are not making classification mistakes, and thus we want our data points from the two classes to lie as far away from each other as possible). This distance is called the margin, so what we want to do is to obtain the maximal margin. Thus, we want to maximize √ 2 wTw , which is equivalent to minimizing √ wTw 2 , which is in turn equivalent to minimizing wTw 2 (since square root is a monotonic function). This quadratic programming problem is expressed as: minw,b wTw 2 subject to: yi(wT xi + b) ≥ 1 (∀ data points xi). 2 Soft-margin extention Consider the case that your data isn’t perfectly linearly separable. For instance, maybe you aren’t guaranteed that all your data points are correctly labelled, so you want to allow some data points of one class to appear on the other side of the boundary. We can introduce slack variables - an i ≥ 0 for each xi . Our quadratic program￾ming problem becomes: minw,b, wTw 2 + C P i i subject to: yi(wT xi + b) ≥ 1 − i and i ≥ 0 (∀ data points xi). 3 Nonlinear decision boundary Mapping your data vectors, xi , into a higher-dimension (even infinite) feature space may make them linearly separable in that space (whereas they may not be linearly separable in the original space). The formulation of the quadratic programming problem is as above, but with all xi replaced with φ(xi), where φ provides the higher-dimensional mapping. So we have the standard SVM formu￾lation: