CrawlCheck

Datasets · by problem

A verification page is being served instead of the file

The server answered with HTTP 200 — so uptime monitors, Search Console and your host all report this as healthy — but the body is a JavaScript verification page, not the file. Crawlers do not run JavaScript on these requests. They receive the interstitial and move on. 79 sites had this problem when we measured them in the last 7 days. The list gives each one with its domain, name where known, platform, every technology we detected, its A–F grade with all its findings, location where known, and what it serves AI crawlers.

  • 79 sites, last 7 days
  • 3 US local businesses
  • 4% also block an AI crawler
  • $99 once, 30 days of downloads

Who has it

Platform

No CMS detected 84%WordPress 11%Next.js 5%

Grade

F 100%

Where

Wider corpus 62%SaaS and software 28%Government and education 6%US local businesses 4%

WhereCategoryGradePlatformAI crawlers
SaaS and softwarePublisherFNo CMS detectedallowed
Seattle, WashingtonCafeFNo CMS detectedallowed
Wider corpusOtherFNo CMS detectedallowed
Wider corpusOtherFNo CMS detectedallowed
Wider corpusOtherFNo CMS detectedallowed
Wider corpusOtherFWordPressallowed
Wider corpusPublisherFWordPressallowed
SaaS and softwareOtherFNo CMS detectedallowed

A sample from the live files. Names and websites are withheld here; they are in the file you buy. A business found with a defect is never named publicly before its owner has been contacted.

Every field: what a row holds · 25 free rows · a problem, country or industry we do not list: order a custom dataset.