# robots.txt for clearscope.io — corrected by CrawlCheck, 2026-09-03 # Source: the file the site served at the time. Every change is listed here; nothing else was touched. # No change needed: the served file had none of the defects this generator repairs. # - Read from https://www.clearscope.io/robots.txt (the apex redirected there); publish at the host you serve. # Verify after publishing: curl -sI https://clearscope.io/robots.txt must show text/plain, on the bare URL, with no cache-buster. # See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-agent: * # Disallow: / User-agent: Clearscopebot Disallow: /legal/*