eTD Explore

Start Over Committee Member Runze Li Committee Member C Lee Giles Keyword web crawler Keyword robots.txt

1. A COMPREHENSIVE STUDY OF THE REGULATION AND BEHAVIOR OF WEB CRAWLERS

open_access

Open Access

Author:: Sun, Yang
Title:: A COMPREHENSIVE STUDY OF THE REGULATION AND BEHAVIOR OF WEB CRAWLERS
Graduate Program:: Information Sciences and Technology
Keywords:: botseer
ethics
bias
web crawler
robots.txt
File:: Download ysun-thesis.pdf
Committee Members:: C Lee Giles, Committee Chair/Co-Chair
James Z Wang, Committee Member
Prasenjit Mitra, Committee Member
Runze Li, Committee Member