Model-based Methods Bayesian Classifier Minimal Error Bayesian Classifier 监督模式识别方法总结 min P(e)=P(elx)p(x)dx P(elx)=P(@Ix)if x isassigned to.i 2009-11-17 .if P(ox)>P(@x).assign x to Bayesian Formula: P(o,Ix)=P(x)P(o) p(x) decision based on p(xo)and P()) The model Several equivalent forms of Bavesian decision rules. Model-based Methods Model-based Methods ▣Bayesian Classifier ▣Bayesian Classifier Minimal Risk Bayesian Classifier Examples under certain distributions Considering not the error only.but the cost of the error Risk: Ra,lx)=∑ao,)Po,x) Model:P(x)=p(x)P() p(x) 时 Decision:a=arg min R(ax) Model-based Methods Model-based Methods Density estimation ▣Density estimation Parametric Estimation Maximum Likelihood Estimation 1(0=p三1)=Πx Nonparametric Estimation ◆Histogram ..The likelihood of getting the observation given 0.0.00.o Parzen Window 。Bayesian E4 imation ks-neighbor estimation Same idea as Bayesian decision Advantage:no distribution assumption Risk R(x)=∫i(0.0p(0xd0 Disadvantage: Large sample size needed i-fooixe No closed-form solutions p01三)=EO p() 31 ML estimation and Bavesian estimation under normal distributions
监督模式识别方法总结 2009-11-17 2 Model-based Methods Bayesian Classifier 3 Model-based Methods Bayesian Classifier 4 Model-based Methods Bayesian Classifier Examples under certain distributions 5 Model-based Methods Density estimation 6 Model-based Methods Density estimation
Direct Methods Direct Methods Linear discriminant functions Linear discriminant functions Fisher linear discriminant Multiple-class problem Perceptron MSE... Direct Methods Direct Methods Generalized linear discriminant functions Nonlinear discriminant functions Support vector machine(SVM) Piecewise linear discriminant functions Nearest neighbor methods I-NN,k-NN P k=1 -1 k=2 k=3 7 k=99 Hard Margin(硬间隔) Soft Margin(软间隔) 叶所情误率 c-1 =σ27 Minimum Distance Classifier Bayesian Decision Optimal Classfier Hecewise Linear (represnetatives Subclass centers classfier based on Mimum distance Methods (1-NN,k-NN) Ne Ne Condensed,...) Nonlinear Discriminant Functions Generalized Linear Support Vector Discriminant Functions Machine (SVM)
7 Direct Methods Linear discriminant functions Fisher linear discriminant Perceptron MSE… 8 Direct Methods Linear discriminant functions Multiple-class problem 9 Direct Methods Generalized linear discriminant functions Support vector machine (SVM) 10 Direct Methods Nonlinear discriminant functions Piecewise linear discriminant functions Nearest neighbor methods 1-NN, k-NN 11 Bayesian Decision Optimal Classfier Fisher,Perceptron, Linear Discriminant MSE,… Minimum Distance Classifier Piecewise Linear Discriminant Functions Mimum distance Subclass centers classfier based on (represnetatives) Nearest neighbor Methods (1-NN,k-NN) Improved NN methods Condensed,…) (quick, edited, Generalized Linear Discriminant Functions Support Vector Machine (SVM) Nonlinear Discriminant Functions I i 2