# robots.txt for seoclarity.net — corrected by CrawlCheck, 2026-11-03 # Source: the file the site served at the time. Every change is listed here; nothing else was touched. # Changes: # - Read from https://www.seoclarity.net/robots.txt (the apex redirected there); publish at the host you serve. # - 2 named groups lacked Disallow lines the * group carries. A crawler obeys only its own most specific group and ignores *, so those rules never applied to it. Copied in. # Verify after publishing: curl -sI https://seoclarity.net/robots.txt must show text/plain, on the bare URL, with no cache-buster. User-agent: BrightEdge Crawler/1.0 Disallow: / Disallow: /_hcms/preview/ Disallow: /hs/manage-preferences/ Disallow: /hs/preferences-center/ Disallow: /*?*hs_preview=* Disallow: /*?*hsCacheBuster=* Disallow: /thank-you Disallow: /thank-you/ Disallow: /thank-you* Disallow: /*thank-you$ Disallow: /seo-platform-essentials/confirmation-lead/thank-you Disallow: /competitive-insights-report-3/ Disallow: /see-it-in-action-enterprise-v1 Disallow: /hubfs/Client%20Stories/Client%20Testimonials.pdf User-agent: OAI-SearchBot Allow: / Disallow: /_hcms/preview/ Disallow: /hs/manage-preferences/ Disallow: /hs/preferences-center/ Disallow: /*?*hs_preview=* Disallow: /*?*hsCacheBuster=* Disallow: /thank-you Disallow: /thank-you/ Disallow: /thank-you* Disallow: /*thank-you$ Disallow: /seo-platform-essentials/confirmation-lead/thank-you Disallow: /competitive-insights-report-3/ Disallow: /see-it-in-action-enterprise-v1 Disallow: /hubfs/Client%20Stories/Client%20Testimonials.pdf User-agent: * Disallow: /thank-you Disallow: /thank-you/ Disallow: /thank-you* Disallow: /*thank-you$ Disallow: /seo-platform-essentials/confirmation-lead/thank-you Disallow: /competitive-insights-report-3/ Disallow: /see-it-in-action-enterprise-v1 Disallow: /hubfs/Client%20Stories/Client%20Testimonials.pdf Sitemap: https://www.seoclarity.net/sitemap.xml Disallow: /_hcms/preview/ Disallow: /hs/manage-preferences/ Disallow: /hs/preferences-center/ Disallow: /*?*hs_preview=* Disallow: /*?*hsCacheBuster=*