CrawlCheck
Rules · fixes by platform

How to fix crawler findings on Squarespace

Where each fix is made on Squarespace, including where Squarespace does not let you make it and what to do instead. Every report on a Squarespace site shows these steps under the finding they fix.

WordPress · Shopify · Webflow · Squarespace · Wix · Next.js · Drupal · Cloudflare

Edit robots.txt

  1. Squarespace writes robots.txt itself and does not let you edit the file.
  2. What you can change: Settings → Crawlers (block search engines, block known AI crawlers) and per-page "Hide from search engines". Those switches add or remove lines in the generated file.
  3. If the finding is about lines you need that Squarespace will not write, the only route is a host that lets you serve your own file.

Fixes these findings: ROBOTS_RULES_SHADOWED, ROBOTS_NO_SITEMAP, ROBOTS_NOT_200, ROBOTS_IS_HTML, ROBOTS_IS_CATCHALL, ROBOTS_DISALLOW_ALL, ROBOTS_CTYPE, ROBOTS_BLOCKED, RENDER_RESOURCE_BLOCKED, INSTRUCTION_CONFLICT, DECLARED_SITEMAP_REFUSED, DECLARED_SITEMAP_BROKEN, SITEMAP_BLOCKED, AI_OPTOUT_SET, BOT_BLOCKED_NOT_CITED, BOT_RULES_SHADOWED_NOT_CITED

Publish a sitemap

  1. Squarespace generates /sitemap.xml automatically; it cannot be edited or turned off.
  2. Pages hidden from search engines are left out of it.

Fixes these findings: NO_SITEMAP_FOUND, SITEMAP_NO_LASTMOD, SITEMAP_IS_HTML, SITEMAP_DECLARES_NOTHING, SITEMAP_UNREADABLE, SITEMAP_ORIGIN_ERROR, ROBOTS_NO_SITEMAP, DECLARED_SITEMAP_BROKEN, DECLARED_SITEMAP_REFUSED

Publish llms.txt

  1. Squarespace cannot host a file at the root. Upload llms.txt (for example as a link file on a hidden page), then map /llms.txt to it with URL Mappings (Settings → Developer tools, Advanced on older sites): /llms.txt -> <file URL> 301.

Fixes these findings: NO_LLMS_TXT, NO_LLMS_TXT_NOT_CITED

Send HSTS

  1. Squarespace sets response headers itself; you cannot add HSTS. If the domain runs through Cloudflare, set it there.

Fixes these findings: HSTS_MISSING, HSTS_PRELOAD_INELIGIBLE, HSTS_PRELOAD_REMOVED

Let crawlers through

  1. Settings → Crawlers: "Block known artificial intelligence crawlers" turns AI crawlers away. Turn it off to let them read the site.

Fixes these findings: CHALLENGE_SERVED_200, CHALLENGE_PINNED_AT_EDGE, ANSWER_ENGINE_REFUSED, UNIFORM_REFUSAL, HOMEPAGE_REFUSED, HOMEPAGE_IS_INTERSTITIAL, ORIGIN_REFUSED_SCANNER, CRAWLER_SERVED_LESS, CRAWLER_REDIRECTED_AWAY, BOT_BLOCKED_NOT_CITED

Serve the text, not just code

  1. Remove code blocks, third-party widgets and code injection you no longer use; Squarespace's own scripts cannot be reduced.

Fixes these findings: PAGE_IS_MOSTLY_CODE, CONTENT_NEEDS_JAVASCRIPT

Fix the structured data

  1. Settings → Business information feeds Squarespace's own markup; correct it there.
  2. Extra JSON-LD goes in Code Injection (Header), on plans that include code injection.

Fixes these findings: ENTITY_COLLISION, ENTITY_NO_COORDINATES, GEO_COUNTRY_MISMATCH, NAP_NAME_DRIFT, NAP_UNDECLARED_LISTING

Clear stale copies

  1. Squarespace's CDN refreshes on its own; if Cloudflare is in front, purge there.

Fixes these findings: STALE_CACHE_SERVED, MACHINE_FILE_CACHE_STALE, MACHINE_FILE_CACHE_SPLIT

Check the fix

Scan the site again on the homepage: a fix counts when the finding is gone from the new report. JSON for tools: /api/v1/fix/platform?platform=squarespace.