Contents 1 Introduction L1 Overview 1.2 Classificat 1.3 Organization of the Book 1.4 Exercises References 667 2 Classification 9 2.1 The Classification Process 2.2 Features 2.3 Training and Learning 16 2.4 Supervised Learning and Algorithm Selection 2.5 Approaches to 2.6 Examples 2 6.2 Classification by Size 2.6.3 More Examples 2. 6.4 Classification of Letters 2.7 Exercises 3 Nonmetric Method 2 3.2 Decision Tree Classifier 3.2. 1 Information, Entropy, and Impurity 3.2.2 Information Gain 3.2.3 Decision Tree Issues 3.3 Rule-Based Classifier 3.4 Other Methods 39 3.5 Exercises 40 ReferencesContents 1 Introduction .......................................... 1 1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Classification . ..................................... 3 1.3 Organization of the Book . . ........................... 6 1.4 Exercises ......................................... 6 References ............................................ 7 2 Classification .......................................... 9 2.1 The Classification Process . ............................ 9 2.2 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 Training and Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.4 Supervised Learning and Algorithm Selection . . . . . . . . . . . . . . 17 2.5 Approaches to Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.6.1 Classification by Shape . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.6.2 Classification by Size . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.6.3 More Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.6.4 Classification of Letters . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3 Nonmetric Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2 Decision Tree Classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2.1 Information, Entropy, and Impurity . . . . . . . . . . . . . . . . 29 3.2.2 Information Gain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2.3 Decision Tree Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.2.4 Strengths and Weaknesses . . . . . . . . . . . . . . . . . . . . . . . 38 3.3 Rule-Based Classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.4 Other Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 ix