Whether a sitemap exists could not be determined — every path was refused
A refusal is not an absence. The paths where a sitemap would live answered with a block, so this scan cannot say whether one exists — and it does not guess. Confirm from a second network, and check server logs grouped by source IP: if the block is aimed at CrawlCheck's address, the finding is about our access; if named crawlers see the same refusal, the sitemap is unreachable to them too. 1,347 sites had this problem when we measured them in the last 7 days. The list gives each one with its domain, name where known, platform, every technology we detected, its A–F grade with all its findings, location where known, and what it serves AI crawlers.
- 1,347 sites, last 7 days
- 357 US local businesses
- 2% also block an AI crawler
- $199 once, 30 days of downloads
Who has it
Platform
No CMS detected 84%WordPress 7%Next.js 6%Drupal 1%Squarespace 1%Shopify 1%
Grade
D 74%F 15%C 11%
Where
Wider corpus 50%US local businesses 27%SaaS and software 16%Government and education 7%
| Where | Category | Grade | Platform | AI crawlers |
|---|---|---|---|---|
| SaaS and software | Other | D | No CMS detected | allowed |
| SaaS and software | Other | D | Next.js | allowed |
| SaaS and software | Other | D | No CMS detected | allowed |
| SaaS and software | Other | D | No CMS detected | allowed |
| SaaS and software | Other | D | No CMS detected | allowed |
| SaaS and software | Other | D | No CMS detected | allowed |
| Government and education | Gov Edu | D | No CMS detected | allowed |
| Apache Junction, Arizona | Government | C | No CMS detected | allowed |
A sample from the live files. Names and websites are withheld here; they are in the file you buy. A business found with a defect is never named publicly before its owner has been contacted.
Every field: what a row holds · 25 free rows · a problem, country or industry we do not list: order a custom dataset.