CrawlCheck

Registry

example.org

Fetched 2026-10-23. Graded D, 55 of 100, 6 findings.

llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable

Is this your site? Claim this record — free, by DNS TXT or a meta tag.

This row is listed and machine-readable, but it is not submitted to search engines: it earned none of the six shelves, so the page states a grade and nothing a reader could not get from the report itself. A shelf or an owner claim changes that on the next fetch.

Get these 6 findings fixed

Every defect this row counts, corrected on the site itself and then re-measured from the outside — done is measured, not asserted. One-time, $500 for one local-business site, thirty days of monitoring included.

Get this fixed — $500

What that covers

Claiming is free and changes nothing but the owner mark. What a licence adds is the record behind this entry: re-measured twice a day, finding age on every defect, change receipts between any two dates. Keep it on the daily record.

What the fetch found

FindingSeverity
No XML sitemap found4 of 5
www and the apex both serve the site instead of one redirecting3 of 5
robots.txt does not return 2003 of 5
No llms.txt1 of 5
HTTPS is served without a Strict-Transport-Security header1 of 5
Visitors and crawlers are being served an old copy of this page1 of 5

Section scores on this fetch

Weakest first. 7 of 15 scored sections at 90 or above.

SectionScore
Authorship and trust signals (E-E-A-T proxies)0
Crawl waste0
Machine-file trust chain0
Schema and entity graph14
Machine layer40
Server and hosting75
Error pages75
Accessibility structure (from the delivered HTML)87
Speed a crawler sees, and field data where it exists100
Payload and render path100
Agent instruction surface100
What each agent receives100
Mobile surface100
Exposed files and secrets100
Internal linking100

Shelves

ShelfStateDefinition
llms.txtnot foundWhat counts, and who has it
agents.mdnot foundWhat counts, and who has it
Media kitnot foundWhat counts, and who has it
Entity graphnot foundWhat counts, and who has it
AI access policynot measuredWhat counts, and who has it
Agent-callablenot foundWhat counts, and who has it

The full report for this fetch

Machine-readable: /api/registry?host=example.org. This row states what was served to CrawlCheck at that time. It is not an endorsement, and it says nothing this scan did not measure.