• Motivation and taxonomy of crawlers • Basic crawlers and implementation issues • Universal crawlers • Preferential (focused and topical) crawlers • Evaluation of preferential crawlers • Crawler ethics and conflicts • New developments: social, collaborative, federated crawlers
