Chapter 2 Basic Discriminants The first classification methods we examine are linear discriminant;particularly,Linear Discriminant Analysis and Fisher's Discriminant Analysis.They are similar in that they both produce linear decision functions that in fact are nearly identical,but the two methods have different assumptions and different approaches.In this chapter the two methods are compared and contrasted. 2.1 Linear Discriminant Analysis for Two Populations Given a pair of known populations,m1 and m2,assume that mI has a probability density function (pdf)fi(x),and similarly,T2 has a pdf of f2(x),where fi(x)f2(x).Then intuitively,a decision function for the two populations would arise from looking at the probability ratio:D(x) f2(x) A new observation x is classified as m if D(x)>1 and T2 if D(x)<1.(For cases where D(x)=1, the vector x is unclassifiable.)Let be the space of all possible observations,and denote the set of whereasR,and similarly the set of whereas Ra.(Denote the set x1 as being Ra.) Such a decision function is simple but effective:by determining from which population x is more likely to have come,one can make quick predictions about its origin.However note that the probability ratio decision function is surely not fool proof:when the probabilities f(x)and f2(x) are close together(or not),there is always a chance that x could be from T2 when fi(x)>f2(x). (Or visa versa.)The conditional probability,P(21),of classifying an observation x as T2 when in fact x∈π1is P(21)=P(x∈R2|T)=五(x)dx. (2.1)Chapter 2 Basic Discriminants The first classification methods we examine are linear discriminant; particularly, Linear Discriminant Analysis and Fisher’s Discriminant Analysis. They are similar in that they both produce linear decision functions that in fact are nearly identical, but the two methods have different assumptions and different approaches. In this chapter the two methods are compared and contrasted. 2.1 Linear Discriminant Analysis for Two Populations Given a pair of known populations, π1 and π2, assume that π1 has a probability density function (pdf) f1(x), and similarly, π2 has a pdf of f2(x), where f1(x) 6= f2(x). Then intuitively, a decision function for the two populations would arise from looking at the probability ratio: D(x) = f1(x) f2(x) . A new observation x is classified as π1 if D(x) > 1 and π2 if D(x) < 1. (For cases where D(x) = 1, the vector x is unclassifiable.) Let Ω be the space of all possible observations, and denote the set of x ∈ Ω where f1(x) f2(x) > 1 as R1, and similarly the set of x ∈ Ω where f1(x) f2(x) < 1 as R2. (Denote the set n x | f1(x) f2(x) = 1o as being R3.) Such a decision function is simple but effective: by determining from which population x is more likely to have come, one can make quick predictions about its origin. However note that the probability ratio decision function is surely not fool proof: when the probabilities f1(x) and f2(x) are close together (or not), there is always a chance that x could be from π2 when f1(x) > f2(x). (Or visa versa.) The conditional probability, P(2|1), of classifying an observation x as π2 when in fact x ∈ π1 is P(2|1) = P(x ∈ R2 | π1) = Z R2 f1(x)dx. (2.1) 7