Nc&IS Text Categorization Peng bo 0/31/2010
Text Categorization PengBo 10/31/2010
本次课大纲 ■ Text Categorization Problem definition Build a classifier Naive Bayes Classifier K-Nearest Neighbor Classifier Evaluation
本次课大纲 ◼ Text Categorization ◼ Problem definition ◼ Build a Classifier ◼ Naïve Bayes Classifier ◼ K-Nearest Neighbor Classifier ◼ Evaluation
Definition Given n实例 instance,x∈X, where X is the instance language or instance space Issue: how to represent text documents 固定的类别集合 categories: C={c1,C2…,Cn} a Determine. The category of X:c(ⅩX)∈C, Where o(x) is a categorization function分类函数 We want to know how to build categorization functions( classifiers分类器”)
Definition ◼ Given: ◼ 实例 instance, xX, where X is the instance language or instance space. ◼ Issue: how to represent text documents. ◼ 固定的类别集合 categories: ◼ C = {c1 , c2 ,…, cn} ◼ Determine: ◼ The category of x : c(x)C, where c(x) is a categorization function 分类函数 ◼ We want to know how to build categorization functions (“classifiers 分类器” )
Text Categorization Examples ssign labels to each document or web-page Labels are most often topics such as Yahoo-categories g,"finance,""sports,""news> world>asia>business abels may be genres e.g, editorials""movie-reviews""news abels may be opinion e.g "llke r "hate "neutral Labels may be domain-specific binary SPAM e.g., "interesting-to-me", not-interesting-to-me' a e.g. ,spam. "not-spam' e.g, contains adult language,, doesnt PRN
Text Categorization Examples Assign labels to each document or web-page: ◼ Labels are most often topics such as Yahoo-categories ◼ e.g., "finance," "sports," "news>world>asia>business" ◼ Labels may be genres ◼ e.g., "editorials" "movie-reviews" "news“ ◼ Labels may be opinion ◼ e.g., “like” , “hate” , “neutral” ◼ Labels may be domain-specific binary ◼ e.g., "interesting-to-me" : "not-interesting-to-me” ◼ e.g., “spam” : “not-spam” ◼ e.g., “contains adult language” :“doesn’t
Classification methods ■人工分类 Manual classification Used by yahoo! Looksmart about com odP, Medline Accurate but expensive to scale ■自动文本分类 Automatic document classification ■基于规则:Hand- coded rule-based systems Spam mail filter, n有监督的学习: Supervised learning of a document- label assignment function No free lunch: requires人工标注的训练集hand- classified training data Note that many commercial systems use a mixture of methods
Classification Methods ◼ 人工分类 Manual classification ◼ Used by Yahoo!, Looksmart, about.com, ODP, Medline ◼ Accurate but expensive to scale ◼ 自动文本分类 Automatic document classification ◼ 基于规则：Hand-coded rule-based systems ◼ Spam mail filter,… ◼ 有监督的学习：Supervised learning of a documentlabel assignment function ◼ No free lunch: requires 人工标注的训练集 handclassified training data ◼ Note that many commercial systems use a mixture of methods
Think about it a How to represent text documents and categories a Vectors regions String Language(models) a How to build categorization functions? Closeness similarity to regions Probability to generate the string/language model
Think about it… ◼ How to represent text documents and categories? ◼ Vectors & Regions ◼ String & Language (models) ◼ How to build categorization functions？ ◼ Closeness/Similarity to regions ◼ Probability to generate the string/language model
Nc&IS K-Nearest Neighbors
K-Nearest Neighbors
Classes in a Vector Space ● Government Science Arts
Classes in a Vector Space Government Science Arts
Classification Using Vector Spaces Each training doc a point(vector)labeled by its topic(= class Hypothesis: docs of the same class form a contiguous region of space We define surfaces to delineate classes in space
Classification Using Vector Spaces ◼ Each training doc a point (vector) labeled by its topic (= class) ◼ Hypothesis: docs of the same class form a contiguous region of space ◼ We define surfaces to delineate classes in space
Test document overnment Similarity hypothesis true in general? ● Government Science Arts
Test Document = Government Government Science Arts Similarity hypothesis true in general?