Chapter 2: Getting to Know Your Data Data objects and attribute types Basic statistical Descriptions of data Data visualization Measuring data Similarity and dissimilarity ■ Summary
2 Chapter 2: Getting to Know Your Data ◼ Data Objects and Attribute Types ◼ Basic Statistical Descriptions of Data ◼ Data Visualization ◼ Measuring Data Similarity and Dissimilarity ◼ Summary
Types of Data Sets Record Relational records Data matrix, e. g. numerical matrix crosstabs Document data: text documents, term frequency vector Document 1 050 602 Transaction data graph and network Document 2 0702100300 World Wide Web Document 3 Social or information networks o,,|2|20。0 Molecular Structures Ordered TD tems Video data: sequence of images Bread, Coke, Milk Temporal data: time-series Beer bread Sequential Data: transaction sequences Beer, Coke, Diaper, Milk Genetic sequence data Spatial, image and multimedia Beer, Bread, Diaper, Milk Spatial data: maps Coke, Diaper, Milk Image data Video data
3 Types of Data Sets ◼ Record ◼ Relational records ◼ Data matrix, e.g., numerical matrix, crosstabs ◼ Document data: text documents: termfrequency vector ◼ Transaction data ◼ Graph and network ◼ World Wide Web ◼ Social or information networks ◼ Molecular Structures ◼ Ordered ◼ Video data: sequence of images ◼ Temporal data: time-series ◼ Sequential Data: transaction sequences ◼ Genetic sequence data ◼ Spatial, image and multimedia: ◼ Spatial data: maps ◼ Image data: ◼ Video data: Document 1 season timeout lost wi n game score ball pla y coach team Document 2 Document 3 3 0 5 0 2 6 0 2 0 2 0 0 7 0 2 1 0 0 3 0 0 1 0 0 1 2 2 0 3 0 TID Items 1 Bread, Coke, Milk 2 Beer, Bread 3 Beer, Coke, Diaper, Milk 4 Beer, Bread, Diaper, Milk 5 Coke, Diaper, Milk
Important Characteristics of Structured Data a Dimensionality Curse of dimensionality a sparsity Only presence counts Resolution Patterns depend on the scale a distribution Centrality and dispersion
4 Important Characteristics of Structured Data ◼ Dimensionality ◼ Curse of dimensionality ◼ Sparsity ◼ Only presence counts ◼ Resolution ◼ Patterns depend on the scale ◼ Distribution ◼ Centrality and dispersion
Data Objects Data sets are made up of data objects a data object represents an entity Examples: sales database: customers store items sales medical database: patients treatments university database: students professors, courses Also called samples, examples, instances, data points, objects, tuples. Data objects are described by attributes Database rows-> data objects columns->attributes
5 Data Objects ◼ Data sets are made up of data objects. ◼ A data object represents an entity. ◼ Examples: ◼ sales database: customers, store items, sales ◼ medical database: patients, treatments ◼ university database: students, professors, courses ◼ Also called samples , examples, instances, data points, objects, tuples. ◼ Data objects are described by attributes. ◼ Database rows -> data objects; columns ->attributes
Attributes Attribute(or dimensions, features, variables) a data field representing a characteristic or feature of a data object E., customer_ID, name address Types nominal Binary Numeric: quantitative Interval-scaled Ratio- scaled
6 Attributes ◼ Attribute (or dimensions, features, variables): a data field, representing a characteristic or feature of a data object. ◼ E.g., customer _ID, name, address ◼ Types: ◼ Nominal ◼ Binary ◼ Numeric: quantitative ◼ Interval-scaled ◼ Ratio-scaled
Attribute Types Nominal: categories, states, or names of things Hair-color=auburn, black, blond, brown, grey red, white marital status, occupation id numbers, zip codes Bina Nominal attribute with only 2 states(0 and 1) Symmetric binary: both outcomes equally important e.g. gender Asymmetric binary: outcomes not equally important e.g. medical test(positive Vs. negative Convention: assign 1 to most important outcome(e.g. HIV positive Ordinal Values have a meaningful order (ranking but magnitude between successive values is not known Size=tsmall, medium, large,, grades army rankings
7 Attribute Types ◼ Nominal: categories, states, or “names of things” ◼ Hair_color = {auburn, black, blond, brown, grey, red, white} ◼ marital status, occupation, ID numbers, zip codes ◼ Binary ◼ Nominal attribute with only 2 states (0 and 1) ◼ Symmetric binary: both outcomes equally important ◼ e.g., gender ◼ Asymmetric binary: outcomes not equally important. ◼ e.g., medical test (positive vs. negative) ◼ Convention: assign 1 to most important outcome (e.g., HIV positive) ◼ Ordinal ◼ Values have a meaningful order (ranking) but magnitude between successive values is not known. ◼ Size = {small, medium, large}, grades, army rankings
Numeric Attribute Types Quantity(integer or real-valued) Interval Measured on a scale of equal-sized units Values have order E.g. temperature in c or F calendar dates No true zero-point Ratio Inherent zero-point We can speak of values as being an order of magnitude larger than the unit of measurement (10K° is twice as high as5K°) e.g temperature in Kelvin, length, counts monetary quantities
8 Numeric Attribute Types ◼ Quantity (integer or real-valued) ◼ Interval ◼ Measured on a scale of equal-sized units ◼ Values have order ◼ E.g., temperature in C˚or F˚, calendar dates ◼ No true zero-point ◼ Ratio ◼ Inherent zero-point ◼ We can speak of values as being an order of magnitude larger than the unit of measurement (10 K˚ is twice as high as 5 K˚). ◼ e.g., temperature in Kelvin, length, counts, monetary quantities
Discrete vs Continuous Attributes Discrete attribute Has only a finite or countably infinite set of values E.g. zip codes profession or the set of words in a collection of documents Sometimes, represented as integer variables Note: Binary attributes are a special case of discrete attributes Continuous Attribute Has real numbers as attribute values E.g. temperature, height or weight Practically, real values can only be measured and represented using a finite number of digits Continuous attributes are typically represented as floating-point variables
9 Discrete vs. Continuous Attributes ◼ Discrete Attribute ◼ Has only a finite or countably infinite set of values ◼ E.g., zip codes, profession, or the set of words in a collection of documents ◼ Sometimes, represented as integer variables ◼ Note: Binary attributes are a special case of discrete attributes ◼ Continuous Attribute ◼ Has real numbers as attribute values ◼ E.g., temperature, height, or weight ◼ Practically, real values can only be measured and represented using a finite number of digits ◼ Continuous attributes are typically represented as floating-point variables
Chapter 2: Getting to Know Your Data Data objects and attribute types Basic statistical Descriptions of data Data visualization Measuring data Similarity and dissimilarity ■ Summary 10
10 Chapter 2: Getting to Know Your Data ◼ Data Objects and Attribute Types ◼ Basic Statistical Descriptions of Data ◼ Data Visualization ◼ Measuring Data Similarity and Dissimilarity ◼ Summary
Basic Statistical Descriptions of Data ■ Motivation To better understand the data: central tendency, variation and spread Data dispersion characteristics median, max, min, quantiles, outliers, variance, eto a Numerical dimensions correspond to sorted intervals Data dispersion analyzed with multiple granularities of precision a Boxplot or quantile analysis on sorted intervals a dispersion analysis on computed measures a Folding measures into numerical dimensions a Boxplot or quantile analysis on the transformed cube 11
11 Basic Statistical Descriptions of Data ◼ Motivation ◼ To better understand the data: central tendency, variation and spread ◼ Data dispersion characteristics ◼ median, max, min, quantiles, outliers, variance, etc. ◼ Numerical dimensions correspond to sorted intervals ◼ Data dispersion: analyzed with multiple granularities of precision ◼ Boxplot or quantile analysis on sorted intervals ◼ Dispersion analysis on computed measures ◼ Folding measures into numerical dimensions ◼ Boxplot or quantile analysis on the transformed cube