Site Design Impact on Robots: An Examination of Search Engine Crawler Behavior at Deep and Wide Websites.

Joan A. Smith Michael L. Nelson

Published in: D Lib Mag. (2008)

Keyphrases

search engine
website
web pages
wide range
cooperative
web search engines
web crawler
information retrieval
web crawling
autonomous robots
design principles
web search
mobile robot
user interface
case study
web documents
browsing behavior