Term Vocabulary and Postings Lists Document Delineation Complications: Format/language Documents being indexed can include docs from many different languages a single index may have to contain terms of several languages a Sometimes a document or its components can contain multiple languages/formats French email with a German pdf attachment What is a unit document? A file? An email?( Perhaps one of many in an mbox An email with 5 attachments? A group of files(ppt or la teX as html pages)Term Vocabulary and Postings Lists 6 Complications: Format/language ▪ Documents being indexed can include docs from many different languages ▪ A single index may have to contain terms of several languages. ▪ Sometimes a document or its components can contain multiple languages/formats ▪ French email with a German pdf attachment. ▪ What is a unit document? ▪ A file? ▪ An email? (Perhaps one of many in an mbox.) ▪ An email with 5 attachments? ▪ A group of files (PPT or LaTeX as HTML pages) Document Delineation