# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-Agent: * Disallow: /ads/ Disallow: /sponsors/ Disallow: /binders/new Disallow: /items/new Disallow: /posts/new Disallow: /users/new Disallow: /articles/new Disallow: /binders/edit Disallow: /items/edit Disallow: /posts/edit Disallow: /users/edit Disallow: /articles/edit Disallow: /signup Disallow: /login Disallow: /logout Disallow: /forgot Disallow: /activate/ Disallow: /suspend/ Disallow: /unsuspend/ Disallow: /destroy/ Disallow: /purge/ Disallow: /reset/ Disallow: /logged_exceptions/