1 Introduction g1(x)=w x+b, where w is an m-dimensional vector and b is a bias term, and if one class is on the positive side of the hyperplane, i.e., 91(x)>0, and the other class is on the negative side, the given problem is said to be linearly separable. 1.1.2 Decision Functions for Multiclass Problems 1.1.2.1 Indirect Decision Functions For an n(> 2)-class problem, suppose we have indirect decision functions gi(x) for classes i. To avoid unclassifiable regions, we classify x into class j given rg where arg returns the subscript with the maximum value of gi (x). If more than one decision function take the same maximum value for x, namely, x is on the class boundary, it is not classifiable In the following we discuss several methods to obtain the direct decision functions for multiclass problems 1.1.2.2 One-Against-All Formulation The first approach is to determine the decision functions by the one-against ll formulation 10. We determine the ith decision function gi(x)(i 1, .., n), so that when x belongs to class i g1(x)>0 and when x belongs to one of the remaining classes g(x)<0. (1.14) When x is given, we classify x into class i if gi(x)>0 and gi(x)<oG+ =l,..., n). But by these decision functions, unclassifiable regions exist when more than one decision function are positive or no decision functions are positive, as seen from Fig. 1.3. To resolve these unclassifiable regions we introduce membership functions in Chapter 34 1 Introduction g1(x) = wx + b, (1.11) where w is an m-dimensional vector and b is a bias term, and if one class is on the positive side of the hyperplane, i.e., g1(x) > 0, and the other class is on the negative side, the given problem is said to be linearly separable. 1.1.2 Decision Functions for Multiclass Problems 1.1.2.1 Indirect Decision Functions For an n(> 2)-class problem, suppose we have indirect decision functions gi(x) for classes i. To avoid unclassifiable regions, we classify x into class j given by j = arg max i=1,...,n gi(x), (1.12) where arg returns the subscript with the maximum value of gi(x). If more than one decision function take the same maximum value for x, namely, x is on the class boundary, it is not classifiable. In the following we discuss several methods to obtain the direct decision functions for multiclass problems. 1.1.2.2 One-Against-All Formulation The first approach is to determine the decision functions by the one-against￾all formulation [10]. We determine the ith decision function gi(x) (i = 1,...,n), so that when x belongs to class i, gi(x) > 0, (1.13) and when x belongs to one of the remaining classes, gi(x) < 0. (1.14) When x is given, we classify x into class i if gi(x) > 0 and gj (x) < 0 (j = i, j = 1,...,n). But by these decision functions, unclassifiable regions exist when more than one decision function are positive or no decision functions are positive, as seen from Fig. 1.3. To resolve these unclassifiable regions we introduce membership functions in Chapter 3.