CrawlCheck

Datasets · by problem

A sitemap lastmod is not a valid date

A lastmod value in the sitemap is not in W3C date format (2026-11-06 or 2026-11-06T09:00:00Z). Crawlers ignore a date they cannot parse, so those pages lose the one change signal a sitemap carries. 308 sites had this problem when we measured them in the last 7 days. The list gives each one with its domain, name where known, platform, every technology we detected, its A–F grade with all its findings, location where known, and what it serves AI crawlers.

  • 308 sites, measured in the last 7 days
  • 110 US local businesses
  • 2026-11-20 last counted, recounted every hour
  • $99 once, about 32¢ a row, 30 days of downloads

At a glance: 64% of these sites show no CMS CrawlCheck could detect, and the most common grade among them is C (52%). How CrawlCheck measures

Who has it

Platform

No CMS detected 64%WordPress 21%Next.js 10%Squarespace 5%

Grade

C 52%A 28%B 14%D 5%F 1%

Where

US local businesses 36%Wider corpus 34%Government and education 15%SaaS and software 15%

WhereCategoryGradePlatformAI crawlers
SaaS and softwareOtherCNo CMS detectedallowed
Government and educationGov EduBWordPressallowed
PennsylvaniaFuelBWordPressallowed
Ambler, PennsylvaniaPlace Of WorshipCNo CMS detectedblocked
Wider corpusOtherANo CMS detectedallowed
Wider corpusGov EduANo CMS detectedallowed
SaaS and softwareOtherANo CMS detectedallowed
Government and educationGov EduANext.jsallowed

A sample from the live files. Names and websites are withheld here; they are in the file you buy. A business found with a defect is never named publicly before its owner has been contacted.

Every field: what a row holds · free sample of this list · a problem, country or industry we do not list: order a custom dataset.