CrawlCheck

Datasets · by problem

Almost nothing a machine receives from this page is readable text

A crawler pays for every byte it fetches and can only quote the text. Under 5% readable content means an answer engine downloads the whole page and comes away with almost nothing it can use — the difference between being quotable and being skipped. The content-ratio row above passes at 20%, so without this a page at 2% and a page at 19% were scored identically. No site in the newest files has it; the list fills as sites with it are measured.

  • 0 sites, last 7 days
  • 0 US local businesses
  • 0% also block an AI crawler
  • $99 once, 30 days of downloads

Who has it

A sample from the live files. Names and websites are withheld here; they are in the file you buy. A business found with a defect is never named publicly before its owner has been contacted.

Every field: what a row holds · 25 free rows · a problem, country or industry we do not list: order a custom dataset.