# robots.txt for surferseo.com — corrected by CrawlCheck, 2026-09-03 # Source: the file the site served at the time. Every change is listed here; nothing else was touched. # Changes: # - Added a Sitemap line: https://surferseo.com/sitemap.xml answered 200 with XML but the file never named it. # - Read from https://surferseo.com/robots.txt/ (the apex redirected there); publish at the host you serve. # Verify after publishing: curl -sI https://surferseo.com/robots.txt must show text/plain, on the bare URL, with no cache-buster. User-Agent: * Disallow: /free-outline-generator/?permalink=* Disallow: /dev/ Disallow: /design-system/ Disallow: /archive/ Disallow: /blog?page= Sitemap: https://surferseo.com/sitemap.xml