# # robots.txt for http://www.salon.com # # check out http://info.webcrawler.com/mak/projects/robots/norobots.html # for more info on what this is all about. # User-agent: * Crawl-delay: 20 Disallow: /RealMedia Disallow: /files User-agent: Slurp Crawl-delay: 30