[Datasets](/datasets) · [by problem](/datasets/problems)

# The ARD manifest is valid but its entries will not be found by search

The ARD manifest validates, but entries are missing representativeQueries (or carry fewer than 2 or more than 5), use a renamed media type, or carry non-standard parameters. The semantic index a registry builds comes from representativeQueries: an entry without them is a valid catalogue row that search never returns. 188 sites had this problem when we measured them in the last 7 days. The list gives each one with its domain, name where known, platform, every technology we detected, its A–F grade with all its findings, location where known, and what it serves AI crawlers.

- **188** sites, measured in the last 7 days
- **54** US local businesses
- **2026-11-20** last counted, recounted every hour
- **$99** once, about 53¢ a row, 30 days of downloads

[Buy the list — $99](/credits?item=prob_ard-manifest-warnings#buy)[Free sample of this list (CSV)](/api/datasets/sample?problem=ard-manifest-warnings)[Other problems](/datasets/problems) · [all lanes, $899/month](/datasets#plan-all)

**At a glance:** 67% of these sites show no CMS CrawlCheck could detect, and the most common grade among them is C (55%). [How CrawlCheck measures](/research)

Who has it

Platform

**No CMS detected** 67% **Next.js** 24% **WordPress** 6% **Webflow** 2% **Shopify** 1%

Grade

**C** 55% **A** 31% **B** 9% **D** 4% **F** 1%

Where

**SaaS and software** 47% **US local businesses** 29% **Wider corpus** 23% **Government and education** 1%

| Where | Category | Grade | Platform | AI crawlers |
| --- | --- | --- | --- | --- |
| SaaS and software | Other | A | WordPress | blocked |
| SaaS and software | Gov Edu | A | No CMS detected | allowed |
| SaaS and software | Other | A | Next.js | allowed |
| Pennsylvania | Restaurant | C | No CMS detected | allowed |
| Wider corpus | Other | C | No CMS detected | allowed |
| SaaS and software | Saas | A | No CMS detected | allowed |
| SaaS and software | Other | B | No CMS detected | blocked |
| Hilton, New York | Dentist | C | No CMS detected | allowed |

A sample from the live files. Names and websites are withheld here; they are in the file you buy. A business found with a defect is never named publicly before its owner has been contacted.

Every field: [what a row holds](/datasets#fields) · [free sample of this list](/api/datasets/sample?problem=ard-manifest-warnings) · a problem, country or industry we do not list: [order a custom dataset](/datasets#order).

We read your site the way crawlers do, from our own network, and every report says exactly what was asked for and what came back. [Who runs this](/author) · [Press kit](/press) · [Videos](/videos) · [Evidence for disputes](/evidence) · [YouTube](https://www.youtube.com/@crawlcheck) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io)

#### How we measure, and our own machine files

CrawlCheck reads every site as itself. Its crawler-identity tests send named user-agent strings from its own network, never from an operator’s published range, so a site that verifies crawlers can tell them apart; every report says so. Results reflect what an anonymous non-browser client receives. [What we collect and how to opt out](/policy) · [how every report is timestamped](/proof) · [who runs this](/author) · [press kit](/press) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io).

**Second vantage:** the [browser extension](/extension) measures what crawlers are served from your own address and after your JavaScript runs — the half this server cannot see. From 1.0.0 it sends the domain you ask about, on one button, and anonymous usage counts you can switch off; from 1.0.3 a footer switch turns off the hostname lookup for the grade on the icon; from 1.0.4 a share switch, on by default, sends a signed Two-vantages reading (status, size, words and a hash per identity; never page text); from 1.0.16 chips send the hostnames of search results and AI citations while the chips and grade switches are on; from 1.0.17 a readings switch, on by default, sends one signed page reading a day (numbers, the hostname and the page address with the query removed; never in incognito; at most 40 a day) and only while the share and grade switches are also on. Earlier versions send nothing.

**Our own machine layer for the agentic web:** [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) · [agents.md](/agents.md) · [SKILL.md](/SKILL.md) · [entitymap.json](/entitymap.json) · [media kit](/.well-known/media-kit.json) · [MCP server card](/.well-known/mcp/server-card.json) · [API catalog](/.well-known/api-catalog) · [OpenAPI](/openapi.json) · [robots.txt](/robots.txt) · [sitemap.xml](/sitemap.xml) · [security.txt](/.well-known/security.txt) — every one named in robots.txt, which is exactly what we check for on yours.

**Pages:** [Keys](/keys) · [/](/) · [Listing check](/listing-check) · [SDK](/sdk) · [Conduct: Google inspectiontool](/agents/conduct/google-inspectiontool) | **Blog:** [Google's Googlebot check passes 198 of 200 rented servers](/blog/googles-googlebot-verification-rule-passes-any-google-cloud-server) | **Glossary:** [training data](/glossary/training-data) · [Retrieval precision](/glossary/retrieval-precision) · [Canonical RDF](/glossary/canonical-rdf) · [MCP resource](/glossary/mcp-resource) · [inclusion proof](/glossary/inclusion-proof) · [Content-Language header](/glossary/content-language)

[Contact](/contact) · [Partnerships](/partnerships) · [Terms of service](/terms) · [Refund policy](/refunds) · [Trust center](/trust) · [Subprocessors](/subprocessors) · [Changelog](/changelog) · [Report an error](/report-error)
