Datasets · by problem

The ARD manifest breaks the specification

The site's ARD manifest (/.well-known/ard.json or the source it advertises) fails the rules every entry must meet: a urn:air:<publisher>:<name> identifier, a displayName, a media type, and exactly one of url or data. A registry rejects the entry, so the resource it describes is not discoverable; where the trust manifest names an identity that does not belong to the URN's publisher, the entry is rejected as namespace squatting. 6,471 sites had this problem when we measured them in the last 7 days. The list gives each one with its domain, name where known, platform, every technology we detected, its A–F grade with all its findings, location where known, and what it serves AI crawlers.

• 6,471 sites, measured in the last 7 days
• 982 US local businesses
• 2026-11-09 last counted, recounted every hour
• $199 once, about 3.1¢ a row, 30 days of downloads

At a glance: 74% of these sites show no CMS CrawlCheck could detect, and the most common grade among them is D (56%). How CrawlCheck measures

Every field: what a row holds · free sample of this list · a problem, country or industry we do not list: order a custom dataset.